Scalable multiscale modeling of platelets with 100 million particles

Han, Changnian; Zhang, Peng; Zhu, Yicong; Cong, Guojing; Kozloski, James R.; Yang, Chih Chieh; Zhang, Leili; Deng, Yuefan

doi:10.1007/s11227-022-04648-4

Title: Scalable multiscale modeling of platelets with 100 million particles

Journal Article · Wed Jun 29 00:00:00 EDT 2022 · Journal of Supercomputing

DOI:https://doi.org/10.1007/s11227-022-04648-4· OSTI ID:1876268

Han, Changnian ^[1]; Zhang, Peng ^[1]; Zhu, Yicong ^[1]; Cong, Guojing ^[2]; Kozloski, James R. ^[3]; Yang, Chih Chieh ^[3]; Zhang, Leili ^[3];

^[4]

Stony Brook Univ., NY (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
IBM Thomas J. Watson Research Center, Yorktown Heights, NY (United States)
Stony Brook Univ., NY (United States); New York University, Abu Dhabi (United Arab Emirates)

Here, we developed the core components of the AI-aided multiple time stepping algorithm for multiscale modeling of cell dynamics. This algorithm was implemented and analyzed on two supercomputer architectures with an application of simulating the aggregation of 250 platelets, or 102 million particles. To scale on these computers with complex memory and network architectures with GPUs, we devised a biomechanics-informed task mapping scheme to optimize load imbalance, communications, and memory utilization. Our simulations, scaling well up to 192 nodes on a Summit-like supercomputer with a peak speed of 11 petaflops, achieved a rate of 423 μs/day which is 500 times faster than the conventional algorithm using static time step and this has enabled studies of record size blood clots at record spatial–temporal resolutions. Additionally, we discovered the sensitive dependence of the scalability and execution time on the methods of decomposition, CPU–GPU coupling, and task mapping.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; SUNY-IBM Consortium

Grant/Contract Number:: AC05-00OR22725; FP00004096

OSTI ID:: 1876268

Journal Information:: Journal of Supercomputing, Vol. TBD, Issue 1; ISSN 0920-8542

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (32)

Smoothed Particle Hydrodynamics Monaghan, J. J. Annual Review of Astronomy and Astrophysics, Vol. 30, Issue 1 https://doi.org/10.1146/annurev.aa.30.090192.002551	journal	September 1992
Molecular dynamics algorithm for multiple time scales: Systems with long range forces Tuckerman, Mark E.; Berne, Bruce J.; Martyna, Glenn J. The Journal of Chemical Physics, Vol. 94, Issue 10 https://doi.org/10.1063/1.460259	journal	May 1991
Heart Disease and Stroke Statistics—2022 Update: A Report From the American Heart Association Tsao, Connie W.; Aday, Aaron W.; Almarzooq, Zaid I. Circulation, Vol. 145, Issue 8 https://doi.org/10.1161/CIR.0000000000001052	journal	February 2022
Blood flow velocity effects and role of activation delay time on growth and form of platelet thrombi Pivkin, I. V.; Richardson, P. D.; Karniadakis, G. Proceedings of the National Academy of Sciences, Vol. 103, Issue 46 https://doi.org/10.1073/pnas.0608546103	journal	November 2006
The CORAL Supercomputer Systems Hanson, William A. IBM Journal of Research and Development https://doi.org/10.1147/JRD.2019.2960220	journal	January 2019
A multiple time stepping algorithm for efficient multiscale modeling of platelets flowing in blood plasma Zhang, Peng; Zhang, Na; Deng, Yuefan Journal of Computational Physics, Vol. 284 https://doi.org/10.1016/j.jcp.2015.01.004	journal	March 2015
The Finite Element Method: Linear Static and Dynamic Finite Element Analysis: Thomas J. R. Hughes Belytschko, Ted Computer-Aided Civil and Infrastructure Engineering, Vol. 4, Issue 3 https://doi.org/10.1111/j.1467-8667.1989.tb00025.x	journal	September 1989
Research approaches for studying flow-induced thromboembolic complications in blood recirculating devices Bluestein, Danny Expert Review of Medical Devices, Vol. 1, Issue 1 https://doi.org/10.1586/17434440.1.1.65	journal	September 2004
SPH on GPU with CUDA Hérault, Alexis; Bilotta, Giuseppe; Dalrymple, Robert A. Journal of Hydraulic Research, Vol. 48, Issue sup1 https://doi.org/10.1080/00221686.2010.9641247	journal	January 2010
The Sunway TaihuLight supercomputer: system and applications Fu, Haohuan; Liao, Junfeng; Yang, Jinzhe Science China Information Sciences, Vol. 59, Issue 7 https://doi.org/10.1007/s11432-016-5588-7	journal	June 2016
Scalable molecular dynamics with NAMD Phillips, James C.; Braun, Rosemary; Wang, Wei Journal of Computational Chemistry, Vol. 26, Issue 16, p. 1781-1802 https://doi.org/10.1002/jcc.20289	journal	January 2005
GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation Hess, Berk; Kutzner, Carsten; van der Spoel, David Journal of Chemical Theory and Computation, Vol. 4, Issue 3 https://doi.org/10.1021/ct700301q	journal	February 2008
Blood flow: multi-scale modeling and visualization Insley, Joseph A.; Grinberg, Leopold; Fedosov, Dmitry A. Proceedings of the 2011 companion on High Performance Computing Networking, Storage and Analysis Companion - SC '11 Companion https://doi.org/10.1145/2148600.2148673	conference	January 2011
Scalable molecular dynamics with NAMD on the IBM Blue Gene/L system Kumar, S.; Huang, C.; Zheng, G. IBM Journal of Research and Development, Vol. 52, Issue 1.2 https://doi.org/10.1147/rd.521.0177	journal	January 2008
Multiscale Particle-Based Modeling of Flowing Platelets in Blood Plasma Using Dissipative Particle Dynamics and Coarse Grained Molecular Dynamics Zhang, Peng; Gao, Chao; Zhang, Na Cellular and Molecular Bioengineering, Vol. 7, Issue 4 https://doi.org/10.1007/s12195-014-0356-5	journal	September 2014
Platelet Adhesive Dynamics. Part I: Characterization of Platelet Hydrodynamic Collisions and Wall Effects Mody, Nipa A.; King, Michael R. Biophysical Journal, Vol. 95, Issue 5 https://doi.org/10.1529/biophysj.107.127670	journal	September 2008
Coarse Grained Model for Semiquantitative Lipid Simulations Marrink, Siewert J.; de Vries, Alex H.; Mark, Alan E. The Journal of Physical Chemistry B, Vol. 108, Issue 2 https://doi.org/10.1021/jp036508g	journal	January 2004
Anton 2: Raising the Bar for Performance and Programmability in a Special-Purpose Molecular Dynamics Supercomputer Shaw, David E.; Grossman, J. P.; Bank, Joseph A. SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.9	conference	November 2014
An Adaptive Load Balancing Method for Parallel Molecular Dynamics Simulations Deng, Yuefan; Peierls, Ronald F.; Rivera, Carlos Journal of Computational Physics, Vol. 161, Issue 1 https://doi.org/10.1006/jcph.2000.6501	journal	June 2000
Multiscale modeling of biomolecular systems: in serial and in parallel Ayton, Gary S.; Noid, Will G.; Voth, Gregory A. Current Opinion in Structural Biology, Vol. 17, Issue 2 https://doi.org/10.1016/j.sbi.2007.03.004	journal	April 2007
Multiscale Modeling of Platelet Adhesion and Thrombus Growth Wang, Weiwei; King, Michael R. Annals of Biomedical Engineering, Vol. 40, Issue 11 https://doi.org/10.1007/s10439-012-0558-8	journal	April 2012
Fast Parallel Algorithms for Short-Range Molecular Dynamics Plimpton, Steve Journal of Computational Physics, Vol. 117, Issue 1 https://doi.org/10.1006/jcph.1995.1039	journal	March 1995
Dissipative particle dynamics: Bridging the gap between atomistic and mesoscopic simulation Groot, Robert D.; Warren, Patrick B. The Journal of Chemical Physics, Vol. 107, Issue 11 https://doi.org/10.1063/1.474784	journal	September 1997
A three-dimensional phase-field model for multiscale modeling of thrombus biomechanics in blood vessels Zheng, Xiaoning; Yazdani, Alireza; Li, He PLOS Computational Biology, Vol. 16, Issue 4 https://doi.org/10.1371/journal.pcbi.1007709	journal	April 2020
A Multiscale Model for Recruitment Aggregation of Platelets by Correlating with In Vitro Results Gupta, Prachi; Zhang, Peng; Sheriff, Jawaad Cellular and Molecular Bioengineering, Vol. 12, Issue 4 https://doi.org/10.1007/s12195-019-00583-2	journal	July 2019
Molecular dynamics simulations of biomolecules Karplus, Martin; McCammon, J. Andrew Nature Structural Biology, Vol. 9, Issue 9 https://doi.org/10.1038/nsb0902-646	journal	September 2002
Platelet Adhesive Dynamics. Part II: High Shear-Induced Transient Aggregation via GPIbα-vWF-GPIbα Bridging Mody, Nipa A.; King, Michael R. Biophysical Journal, Vol. 95, Issue 5 https://doi.org/10.1529/biophysj.107.128520	journal	September 2008
A Multiscale Red Blood Cell Model with Accurate Mechanics, Rheology, and Dynamics Fedosov, Dmitry A.; Caswell, Bruce; Karniadakis, George Em Biophysical Journal, Vol. 98, Issue 10 https://doi.org/10.1016/j.bpj.2010.02.002	journal	May 2010
Artificial intelligence for accelerating time integrations in multiscale modeling Han, Changnian; Zhang, Peng; Bluestein, Danny Journal of Computational Physics, Vol. 427 https://doi.org/10.1016/j.jcp.2020.110053	journal	February 2021
Biomolecular Simulation: A Computational Microscope for Molecular Biology Dror, Ron O.; Dirks, Robert M.; Grossman, J. P. Annual Review of Biophysics, Vol. 41, Issue 1 https://doi.org/10.1146/annurev-biophys-042910-155245	journal	June 2012
Efficient and Large-Scale Dissipative Particle Dynamics Simulations on GPU Yang, Keda; Bai, Zhiqiang; Su, Jiaye Soft Materials, Vol. 12, Issue 2 https://doi.org/10.1080/1539445X.2013.858359	journal	January 2014
Anton, a special-purpose machine for molecular dynamics simulation Shaw, David E.; Chao, Jack C.; Eastwood, Michael P. Communications of the ACM, Vol. 51, Issue 7 https://doi.org/10.1145/1364782.1364802	journal	July 2008

Similar Records

Scalable All-pairs Shortest Paths for Huge Graphs on Multi-GPU Clusters

Conference · Mon Jun 01 00:00:00 EDT 2020 · OSTI ID:1876268

Sao, Piyush; Lu, Hao; Kannan, Ramakrishnan {Ramki}; +3 more

GSoFa: Scalable Sparse Symbolic LU Factorization on GPUs

Journal Article · Fri Apr 01 00:00:00 EDT 2022 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1876268

Gaihre, Anil; Li, Xiaoye Sherry; Liu, Hang

Scalable molecular dynamics with NAMD on the Summit system

Journal Article · Thu Nov 01 00:00:00 EDT 2018 · IBM Journal of Research and Development · OSTI ID:1876268

Acun, B.; Hardy, D. J.; Kale, L. V.; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING
scalability
multiscale modeling
artificial intelligence
load balance

Title: Scalable multiscale modeling of platelets with 100 million particles

Citation Formats

References (32)

Similar Records

Related Subjects