skip to main content
10.1145/1654059.1654121acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

Beyond homogeneous decomposition: scaling long-range forces on Massively Parallel Systems

Published: 14 November 2009 Publication History

Abstract

With supercomputers anticipated to expand from thousands to millions of cores, one of the challenges facing scientists is how to effectively utilize this ever-increasing number. We report here an approach that creates a heterogeneous decomposition by partitioning effort according to the scaling properties of the component algorithms. We demonstrate our strategy by developing a capability to model hot dense plasma. We have performed benchmark calculations ranging from millions to billions of charged particles, including a 2.8 billion particle simulation that achieved 259.9 TFlop/s (26% of peak performance) on the 294,912 cpu JUGENE computer at the Jülich Supercomputing Centre in Germany. With this unprecedented simulation capability we have begun an investigation of plasma fusion physics under conditions where both theory and experiment are lacking--in the strongly-coupled regime as the plasma begins to burn.
Our strategy is applicable to other problems involving long-range forces (i.e., biological or astrophysical simulations). We believe that the flexible heterogeneous decomposition approach demonstrated here will allow many problems to scale across current and next-generation machines.

References

[1]
M. P. Allen and D. J. Tildesley. Computer Simulation of Liquids. Oxford University Press, New York, 1989.
[2]
F. F. Chen. Introduction to Plasma Physics and Controlled Fusion. Plenum Press, New York, 1984.
[3]
E. Chow, C. A. Rendleman, K. J. Bowers, R. O. Dror, D. H. Hughes, J. Gullingsrud, F. D. Sacerdoti, and D. E. Shaw. Desmond performance on a cluster of multicore processors. Technical Report DESRES/TR-2008-01, D. E. Shaw Research, July 2008. http://www.deshawresearch.com/resources_desmondpublications.html.
[4]
M. Deserno and C. Holm. How to mesh up Ewald sums. I. a theoretical and numerical comparison of various particle mesh routines. Journal of Chemical Physics, 109(18):7678--93, 1998.
[5]
M. Deserno and C. Holm. How to mesh up Ewald sums. II. An accurate error estimate for the particle-particle-particle-mesh algorithm. J. Chem. Phys., 109:7694, 1998.
[6]
M. Eleftheriou, B. Fitch, A. Rayshubskiy, T. J. C. Ward, and R. S. Germain. Performance measurements of the 3d FFT on the Blue Gene/L supercomputer. In Euro-Par 2005 Parallel Processing: 11th International Euro-Par Conference, Lecture Notes in Computer Science, volume 3648, pages 795--803, Lisbon, Portugal, 2005. Springer-Verlag.
[7]
M. Eleftheriou, B. G. Fitch, A. Rayshubskiy, T. J. C. Ward, and R. S. Germain. Scalable framework for 3d FFTs on the Blue Gene/L supercomputer: Implementation and early performance measurements. IBM Journal of Research and Development, 49(2--3):457--464, Mar--May 2005.
[8]
http://www.fftw.org/.
[9]
B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, T. J. C. Ward, M. Giampapa, M. C. Pitman, and R. S. Germain. Progress in scaling biomolecular simulations to petaflop scale platforms. In Lehner, W and Meyer, N and Streit, A and Stewart, C, editor, Euro-Par 2006 Workshops: Parallel Processing, volume 4375 of Lecture Notes in Computer Science, pages 279--288, Heidelberger Platz 3, D-14197 Berlin, Germany, 2007. Inst Sci Comp; Inst Syst Architecture, Springer-Verlag Berlin. 12th Euro-Par 2006 Workshops, Dresden, Germany, Aug 29--Sep 01, 2006.
[10]
B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, T. J. C. Ward, M. E. Giampapa, M. C. Pitman, J. W. Pitera, W. C. Swope, and R. S. Germain. Blue Matter: Scaling of N-body simulations to one atom per node. IBM Journal of Research and Development, 52(1--2):145--158, Jan--Mar 2008.
[11]
J. N. Glosli, D. F. Richards, K. J. Caspersen, R. E. Rudd, J. A. Gunnels, and F. H. Streitz. Extending stability beyond cpu millennium: a micron-scale atomistic simulation of kelvin-helmholtz instability. In SC '07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing, pages 1--11, New York, NY, USA, 2007. ACM.
[12]
Grape newletter vol. 8 (dec 16 2006), http://grape.mtk.nao.ac.jp/grape/newsletter/061216.html.
[13]
R. W. Hockney and J. W. Eastwood. Computer Simulation Using Particles. McGraw-Hill, New York, 1981.
[14]
M. H. Key. Status of and prospects for the fast ignition inertial fusion concept. Physics of Plasmas, 14:055502, 2007.
[15]
G. Kikugawa, R. Apostolov, N. Kamiya, M. Taiji, R. Himeno, H. Nakamura, and Y. Yonezawa. Application of mdgrape-e, a special purpose board for molecular dynamics simulations, to periodic biomolecular systems. J. Comput. Chem., 30(1):110--118, 2009.
[16]
S. Kumar, C. Huang, G. Zheng, E. Bohm, A. Bhatele, J. C. Phillips, H. Yu, and L. V. Kalé. Scalable Molecular Dynamics with NAMD on Blue Gene/L. IBM Journal of Research and Development: Applications of Massively Parallel Systems, 52(1/2):177--187, 2008.
[17]
LAMMPS benchmarks. http://lammps.sandia.gov/bench.html.
[18]
J. Makino. The GRAPE project. Comput. Sci. Eng., 8(1):30--40, JAN--FEB 2006.
[19]
E. L. Pollock and J. Glosli. Comments on P3M, FMM, and the Ewald method for large periodic coulombic systems. Computer Physics Communications, 95(2--3):93--110, June 1996.
[20]
C. Sagui and T. Darden. Molecular dynamics simulations of biomolecules. Annu. Rev. Biophys. Biomol. Struct., 28:155--179, 1999.
[21]
C. Sagui and T. Darden. Multigrid methods for classical molecular dynamics simulations of biomolecules. J Chem Phys, 114(15):6578--6591, 2001.
[22]
Y. Shan, J. L. Klepeis, M. P. Eastwood, R. O. Dror, and D. E. Shaw. Gaussian split Ewald: A fast Ewald mesh method for molecular simulation. J. Chem. Phys., 122(5), 2005.
[23]
The IBM Blue Gene Team. Overview of the IBM Blue Gene/P project. IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 52(1--2):199--220, JAN--MAR 2008.

Cited By

View all
  • (2025)An interpretable DeePMD-kit performance model for emerging supercomputersCCF Transactions on High Performance Computing10.1007/s42514-024-00209-8Online publication date: 17-Feb-2025
  • (2023)VVQ: Virtualizing Virtual Channel for Cost-Efficient Protocol Deadlock Avoidance2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA56546.2023.10071059(1072-1084)Online publication date: Feb-2023
  • (2020)Performance drop at executing communication-intensive parallel algorithmsThe Journal of Supercomputing10.1007/s11227-019-03142-8Online publication date: 6-Jan-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
November 2009
778 pages
ISBN:9781605587448
DOI:10.1145/1654059
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2009

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

Conference

SC '09
Sponsor:

Acceptance Rates

SC '09 Paper Acceptance Rate 59 of 261 submissions, 23%;
Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)1
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)An interpretable DeePMD-kit performance model for emerging supercomputersCCF Transactions on High Performance Computing10.1007/s42514-024-00209-8Online publication date: 17-Feb-2025
  • (2023)VVQ: Virtualizing Virtual Channel for Cost-Efficient Protocol Deadlock Avoidance2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA56546.2023.10071059(1072-1084)Online publication date: Feb-2023
  • (2020)Performance drop at executing communication-intensive parallel algorithmsThe Journal of Supercomputing10.1007/s11227-019-03142-8Online publication date: 6-Jan-2020
  • (2018)TCEPProceedings of the 45th Annual International Symposium on Computer Architecture10.1109/ISCA.2018.00065(712-725)Online publication date: 2-Jun-2018
  • (2016)Contention-based congestion management in large-scale networksThe 49th Annual IEEE/ACM International Symposium on Microarchitecture10.5555/3195638.3195674(1-13)Online publication date: 15-Oct-2016
  • (2016)Evaluating and extending user-level fault tolerance in MPI applicationsThe International Journal of High Performance Computing Applications10.1177/109434201562362330:3(305-319)Online publication date: 27-Jul-2016
  • (2016)Contention-based congestion management in large-scale networks2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO.2016.7783733(1-13)Online publication date: Oct-2016
  • (2016)Multiple Staggered Mesh Ewald: Boosting the Accuracy of the Smooth Particle Mesh Ewald MethodJournal of Chemical Theory and Computation10.1021/acs.jctc.6b0070112:11(5596-5608)Online publication date: 28-Oct-2016
  • (2015)Overcoming far-end congestion in large-scale networks2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA)10.1109/HPCA.2015.7056051(415-427)Online publication date: Feb-2015
  • (2014)Evaluating User-Level Fault Tolerance for MPI ApplicationsProceedings of the 21st European MPI Users' Group Meeting10.1145/2642769.2642775(57-62)Online publication date: 9-Sep-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media