skip to main content
10.1145/3458817.3476222acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

Enabling large-scale correlated electronic structure calculations: scaling the RI-MP2 method on summit

Published: 13 November 2021 Publication History

Abstract

Second-order Møller-Plesset perturbation theory using the Resolution-of-the-Identity approximation (RI-MP2) is a state-of-the-art approach to accurately estimate many-body electronic correlation effects. This is critical for predicting the physicochemical properties of complex molecular systems; however, the scale of these calculations is limited by their extremely high computational cost. In this paper, a novel many-GPU algorithm and implementation of a molecular-fragmentation-based RI-MP2 method are presented that enable correlated calculations on over 180,000 electrons and 45,000 atoms using up to the entire Summit supercomputer in 12 minutes. The implementation demonstrates remarkable speedups with respect to other current GPU and CPU codes, excellent strong scalability on Summit achieving 89.1% parallel efficiency on 4600 nodes, and shows nearly-ideal weak scaling up to 612 nodes. This work makes feasible ab initio correlated quantum chemistry calculations on significantly larger molecular scales than before on both large supercomputing systems and on commodity clusters, with a potential for major impact on progress in chemical, physical, biological and engineering sciences.

Supplementary Material

MP4 File (Enabling Large-Scale Correlated Electronic Structure Calculations_ Scaling the RI-MP2 Method on Summit.mp4.mp4)
Presentation video

References

[1]
C. Møller and M. S. Plesset. Note on an approximation treatment for many-electron systems. Phys. Rev., 46:618--622, 1934.
[2]
R. J. Bartlett and D. M. Silver. Correlation energy in lih, bh, and hf with many-body perturbation theory using slater-type atomic orbitals. Int. J. Quantum Chem., Symp., 8:271--276, 1974.
[3]
N. C. Handy, P. J. Knowles, and K. Somasundram. On the convergence of the møller-plesset perturbation series. Theor. Chim. Acta, 68:87, 1985.
[4]
W. D. Laidig, G. Fitzgerald, and R. J. Bartlett. Is fifth-order mbpt enough? Chem. Phys. Lett., 113:151--158, 1985.
[5]
P. M. W. Gill and L. Radom. Deceptive convergence in møller-plesset perturbation energies. Chem. Phys. Lett., 132:16--22, 1986.
[6]
P. M. W. Gill, J. A. Pople, L. Radom, and R. H. Nobes. Why does unrestricted møller-plesset perturbation theory converge so slowly for spin-contaminated wave functions? J. Chem. Phys., 89:7307--7314, 1988.
[7]
J. Olsen, O. Christiansen, H. Koch, and P. Jorgensen. Surprising cases of divergent behavior in møller-plesset perturbation theory. J. Chem. Phys., 105:5082--5090, 1996.
[8]
F. H. Stillinger. Møller-plesset convergence issues in computational quantum chemistry. J. Chem. Phys., 112:9711--9715, 2000.
[9]
M. L. Leininger, W. D. Allen, H. F. Schaefer, and C. D. Sherrill. Is møller-plesset perturbation theory a convergent ab initio method? J. Chem. Phys., 112:9213--9222, 2000.
[10]
M. Feyereisen, G. Fitzgerald, and A. Komornicki. Use of approximate integrals in ab initio theory. an application in mp2 energy calculations. Chem. Phys. Lett., 208:359--363, 1993.
[11]
Jan Almlöf. Elimination of energy denominators in møller-plesset perturbation theory by a laplace transform approach. Chem. Phys. Lett., 181(4):319--320, 1991.
[12]
Marco Häser. Møller-plesset (mp2) perturbation theory for large molecules. Theor. Chim. Acta, 87:147--173, 1993.
[13]
Todd J. Martinez and Emily A. Carter. Pseudospectral møller-plesset perturbation theory through third order. J. Chem. Phys., 100:3631--3638, 1994.
[14]
G. Hetzer, P. Pulay, and H. J. Werner. Multipole approximation of distant pair energies in local mp2 calculations. Chem. Phys. Lett., 290:143--149, 1998.
[15]
Mark S. Gordon, Giuseppe Barca, Sarom S. Leang, David Poole, Alistair P. Rendell, Jorge L. Galvez Vallejo, and Bryce Westheimer. Novel computer architectures and quantum chemistry. J. Phys. Chem. A, 124(23):4557--4582, 06 2020. URL https://doi.org/10.1021/acs.jpca.0c02249.
[16]
David E. Bernholdt and Robert J. Harrison. Large-scale correlated electronic structure calculations: the ri-mp2 method on parallel computers. Chemical Physics Letters, 250(5):477--484, 1996. ISSN 0009-2614. URL https://www.sciencedirect.com/science/article/pii/0009261496000541.
[17]
Michio Katouda, Masato Kobayashi, Hiromi Nakai, and Shigeru Nagase. Two-level hierarchical parallelization of second-order møller-plesset perturbation calculations in divide-and-conquer method. J. Comput. Chem., 32:2756--2764, 2011.
[18]
Mauro Del Ben, Jürg Hutter, and Joost VandeVondele. Second-order møller-plesset perturbation theory in the condensed phase: An efficient and massively parallel gaussian and plane waves approach. J. Chem. Theory Comput., 8:4177--4188, 2012.
[19]
Kasper Kristensen, Thomas Kjaergaard, Ida-Marie Høyvik, Patrick Ettenhuber, Poul Jørgensen, Branislav Jansik, Simen Reine, and Jacek Jakowski. The divide-expand-consolidate mp2 scheme goes massively parallel. Mol. Phys., 111:1196--1201, 2013.
[20]
Michio Katouda and Takahito Nakajima. Mpi/openmp hybrid parallel algorithm of resolution of identity second-order møller-plesset perturbation calculation for massively parallel multicore supercomputers. J. Chem. Theory Comput., 9: 5373--5380, 2013.
[21]
Hans-Joachim Werner, Gerald Knizia, Christine Krause, Max Schwilk, and Mark Dornbach. Scalable electron correlation methods i.: Pno-lmp2 with linear scaling in the molecular size and near-inverse-linear scaling in the number of processors. J. Chem. Theory Comput., 11:484--507, 2015.
[22]
Michio Katouda, Akira Naruse, Yukihiko Hirano, and Takahito Nakajima. Massively parallel algorithm and implementation of ri-mp2 energy calculation for peta-scale many-core supercomputers. J. Comput. Chem., 37:2623--2633, 2016.
[23]
Tobias Schäfer, Benjamin Ramberger, and Georg Kresse. Quartic scaling mp2 for solids: A highly parallelized algorithm in the plane wave basis. J. Chem. Phys., 146:104101, 2017.
[24]
Luis Angel Martinez-Martinez and Carlos Amador-Bedolla. Gpu algorithm for the scaled opposite-spin (sos) mp2 energy evaluation. J. Mex. Chem. Soc., 61: 60--66, 2017.
[25]
Buu Q. Pham and Mark S. Gordon. Hybrid distributed/shared memory model for the ri-mp2 method in the fragment molecular orbital framework. Journal of Chemical Theory and Computation, 15(10):5252--5258, 2019. URL https://doi.org/10.1021/acs.jctc.9b00409. 31509402.
[26]
Giuseppe M. J. Barca, Simon C. McKenzie, Nathaniel J. Bloomfield, Andrew T. B. Gilbert, and Peter M. W. Gill. Q-mp2-os: Møller-plesset correlation energy by quadrature. Journal of Chemical Theory and Computation, 16(3):1568--1577, 2020. URL https://doi.org/10.1021/acs.jctc.9b01142. 31972086.
[27]
Thomas Kjærgaard, Pablo Baudin, Dmytro Bykov, Janus Juul Eriksen, Patrick Ettenhuber, Kasper Kristensen, Jeff Larkin, Dmitry Liakh, Filip Pawlowski, Aaron Vose, Yang Min Wang, and Poul Jørgensen. Massively parallel and linear-scaling algorithm for second-order møller-plesset perturbation theory applied to the study of supramolecular wires. Computer Physics Communications, 212:152--160, 2017. ISSN 0010-4655. URL https://www.sciencedirect.com/science/article/pii/S0010465516303447.
[28]
Jakub Kollar and Vladimir Frecer. How accurate is the description of ligand-protein interactions by a hybrid qm/mm approach? Journal of Molecular Modeling, 24:11, 2018. URL https://doi.org/10.1007/s00894-017-3537-z.
[29]
Peter M.W. Gill. Molecular integrals over gaussian basis functions. volume 25 of Advances in Quantum Chemistry, pages 141 -- 205. Academic Press, 1994. URL http://www.sciencedirect.com/science/article/pii/S0065327608600192.
[30]
Giuseppe M. J. Barca and Peter M. W. Gill. Two-electron integrals over gaussian geminals. Journal of Chemical Theory and Computation, 12(10):4915--4924, 2016. URL https://doi.org/10.1021/acs.jctc.6b00770. 27598837.
[31]
Giuseppe M. J. Barca, Pierre-François Loos, and Peter M. W. Gill. Many-electron integrals over gaussian basis functions. i. recurrence relations for three-electron integrals. Journal of Chemical Theory and Computation, 12(4):1735--1740, 2016. URL https://doi.org/10.1021/acs.jctc.6b00130. 26981747.
[32]
Giuseppe M. J. Barca and Pierre-François Loos. Three- and four-electron integrals involving gaussian geminals: Fundamental integrals, upper bounds, and recurrence relations. The Journal of Chemical Physics, 147(2):024103, 2017. URL https://doi.org/10.1063/1.4991733.
[33]
Giuseppe M.J. Barca and Pierre-François Loos. Chapter eight - recurrence relations for four-electron integrals over gaussian basis functions. In Philip E. Hoggan, editor, Novel Electronic Structure Theory: General Innovations and Strongly Correlated Systems, volume 76 of Advances in Quantum Chemistry, pages 147--165. Academic Press, 2018. URL https://www.sciencedirect.com/science/article/pii/S006532761730031X.
[34]
Mark S Gordon, Dmitri G Fedorov, Spencer R Pruitt, and Lyudmila V Slipchenko. Fragmentation Methods: A Route to Accurate Calculations on Large Systems. Chemical Reviews, 112(1):632--672, jan 2012. ISSN 0009-2665. URL https://doi.org/10.1021/cr200093j.
[35]
Dmitri G. Fedorov. A combined effective fragment potential-fragment molecular orbital method. II. Analytic gradient and application to the geometry optimization of solvated tetraglycine and chignolin. The Journal of Chemical Physics, 134(3): 034110, January 2011. ISSN 0021-9606, 1089--7690. URL http://aip.scitation.org/doi/10.1063/1.3517110.
[36]
Giuseppe M. J. Barca, David L. Poole, Jorge L. Galvez Vallejo, Melisa Alkan, Colleen Bertoni, Alistair P. Rendell, and Mark S. Gordon. Scaling the hartree-fock matrix build on summit. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '20. IEEE Press, 2020. ISBN 9781728199986.
[37]
Giuseppe M. J. Barca, Jorge L. Galvez-Vallejo, David L. Poole, Alistair P. Rendell, and Mark S. Gordon. High-performance, graphics processing unit-accelerated fock build algorithm. Journal of Chemical Theory and Computation, 16(12):7232--7238, 2020. URL https://doi.org/10.1021/acs.jctc.0c00768. 33206515.
[38]
Giuseppe M. J. Barca, Colleen Bertoni, Laura Carrington, Dipayan Datta, Nuwan De Silva, J. Emiliano Deustua, Dmitri G. Fedorov, Jeffrey R. Gour, Anastasia O. Gunina, Emilie Guidez, Taylor Harville, Stephan Irle35061, Joe Ivanic, Karol Kowalski, Sarom S. Leang, Hui Li, Wei Li, Jesse J. Lutz, Ilias Magoulas, Joani Mato, Vladimir Mironov, Hiroya Nakata, Buu Q. Pham, Piotr Piecuch, David Poole, Spencer R. Pruitt, Alistair P. Rendell, Luke B. Roskop, Klaus Ruedenberg, Tosaporn Sattasathuchana, Michael W. Schmidt, Jun Shen, Lyudmila Slipchenko, Masha Sosonkina, Vaibhav Sundriyal, Ananta Tiwari, Jorge L. Galvez Vallejo, Bryce Westheimer, Marta Włoch, Peng Xu, Federico Zahariev, and Mark S. Gordon. Recent developments in the general atomic and molecular electronic structure system. The Journal of Chemical Physics, 152(15):154102, 2020. URL https://doi.org/10.1063/5.0005188.
[39]
Tait Takatani, Edward G. Hohenstein, Massimo Malagoli, Michael S. Marshall, and C. David Sherrill. Basis set consistent revision of the S22 test set of noncovalent interaction energies. The Journal of Chemical Physics, 132(14):144104, April 2010. ISSN 0021-9606, 1089--7690. URL http://aip.scitation.org/doi/10.1063/1.3378024.
[40]
Michael A. Collins and Ryan P. A. Bettens. Energy-based molecular fragmentation methods. Chemical Reviews, 115(12):5607--5642, 06 2015. URL https://doi.org/10.1021/cr500455b.

Cited By

View all
  • (2024)Automatic molecular fragmentation by evolutionary optimisationJournal of Cheminformatics10.1186/s13321-024-00896-z16:1Online publication date: 19-Aug-2024
  • (2024)High-Performance, Accurate Large-Scale Quantum Chemistry Calculations on GPU Supercomputers using Coulomb-Perturbed FragmentationProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673087(1092-1102)Online publication date: 12-Aug-2024
  • (2024)Advanced Techniques for High-Performance Fock Matrix Construction on GPU ClustersJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0099420:23(10424-10442)Online publication date: 25-Nov-2024
  • Show More Cited By

Index Terms

  1. Enabling large-scale correlated electronic structure calculations: scaling the RI-MP2 method on summit
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          SC '21: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
          November 2021
          1493 pages
          ISBN:9781450384421
          DOI:10.1145/3458817
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          In-Cooperation

          • IEEE CS

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 13 November 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. GPU
          2. MP2
          3. SCF
          4. quantum chemistry
          5. summit

          Qualifiers

          • Research-article

          Funding Sources

          • Department of Energy Exascale Computing Project

          Conference

          SC '21
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

          Upcoming Conference

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)51
          • Downloads (Last 6 weeks)3
          Reflects downloads up to 03 Jan 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Automatic molecular fragmentation by evolutionary optimisationJournal of Cheminformatics10.1186/s13321-024-00896-z16:1Online publication date: 19-Aug-2024
          • (2024)High-Performance, Accurate Large-Scale Quantum Chemistry Calculations on GPU Supercomputers using Coulomb-Perturbed FragmentationProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673087(1092-1102)Online publication date: 12-Aug-2024
          • (2024)Advanced Techniques for High-Performance Fock Matrix Construction on GPU ClustersJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0099420:23(10424-10442)Online publication date: 25-Nov-2024
          • (2024)Multi-GPU RI-HF Energies and Analytic Gradients─Toward High-Throughput Ab Initio Molecular DynamicsJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0087720:17(7503-7515)Online publication date: 28-Aug-2024
          • (2024)An Efficient RI-MP2 Algorithm for Distributed Many-GPU ArchitecturesJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0081420:21(9394-9406)Online publication date: 18-Oct-2024
          • (2023)High-Performance Evaluation of High Angular Momentum 4-Center Gaussian Integrals on Modern Accelerated ProcessorsThe Journal of Physical Chemistry A10.1021/acs.jpca.3c04574127:51(10889-10895)Online publication date: 13-Dec-2023
          • (2023)The General Atomic and Molecular Electronic Structure System (GAMESS): Novel Methods on Novel ArchitecturesJournal of Chemical Theory and Computation10.1021/acs.jctc.3c0037919:20(7031-7055)Online publication date: 4-Oct-2023
          • (2023)Toward Pair Atomic Density Fitting for Correlation Energies with Benchmark AccuracyJournal of Chemical Theory and Computation10.1021/acs.jctc.2c0120119:5(1499-1516)Online publication date: 14-Feb-2023
          • (2023)Memory-Efficient Recursive Evaluation of 3-Center Gaussian IntegralsJournal of Chemical Theory and Computation10.1021/acs.jctc.2c0099519:6(1698-1710)Online publication date: 14-Mar-2023
          • (2022)Scaling correlated fragment molecular orbital calculations on summitProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571894(1-14)Online publication date: 13-Nov-2022
          • Show More Cited By

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media