ABSTRACT
Analysis of scientific simulation data enables scientists to glean insight from simulations. In situ analysis, which can be simultaneously executed with the simulation, mitigates I/O bottlenecks and can accelerate discovery of new phenomena. However, in typical modes of operation, this requires either stalling simulation during analysis phase or transferring data for analysis. We study the scalability challenges of time- and space-shared modes of analyzing large-scale molecular dynamics simulations. We also propose topology-aware mapping for simulation and analysis. We demonstrate the benefits of our approach using LAMMPS code on two supercomputers.
- LAMMPS Documentation. http://lammps.sandia.gov/doc/Manual.html.Google Scholar
- ParaView Catalyst. http://catalyst.paraview.org.Google Scholar
- Cray Performance Measurement and Analysis Tools. Technical Report S-2376-63, 2015.Google Scholar
- Hasan Abbasi, Matthew Wolf, Greg Eisenhauer, Scott Klasky, Karsten Schwan, and Fang Zheng. DataStager: scalable data staging services for petascale applications. Cluster Computing, 13(3):277--290, 2010. Google ScholarDigital Library
- M. P. Allen and D. J. Tildesley. Computer Simulation of Liquids. Oxford Science Publications, 1989.Google ScholarDigital Library
- U. Ayachit, A. Bauer, E. P. N. Duque, G. Eisenhauer, N. Ferrier, J. Gu, K. E. Jansen, B. Loring, Z. Lukic, S. Menon, D. Morozov, P. O'Leary, R. Ranjan, M. Rasquin, C. P. Stone, V. Vishwanath, G. H. Weber, B. Whitlock, M. Wolf, K. J. Wu, and E. W. Bethel. Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures. In SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 921--932, Nov 2016. Google ScholarCross Ref
- A. C. Bauer, H. Abbasi, J. Ahrens, H. Childs, B. Geveci, S. Klasky, K. Moreland, P. O'Leary, V. Vishwanath, B. Whitlock, and E. W. Bethel. In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms. Computer Graphics Forum, 35(3):577--597, 2016. Google ScholarDigital Library
- J.C. Bennett, H. Abbasi, P.-T. Bremer, R. Grout, A. Gyulassy, Tong Jin, S. Klasky, H. Kolla, M. Parashar, V. Pascucci, P. Pebay, D. Thompson, Hongfeng Yu, Fan Zhang, and J. Chen. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012. Google ScholarDigital Library
- A. Bhatele, G. R. Gupta, L. V. Kale, and I. H. Chung. Automated mapping of regular communication graphs on mesh interconnects. In 2010 International Conference on High Performance Computing, Dec 2010. Google ScholarCross Ref
- Abhinav Bhatele, Todd Gamblin, Steven H Langer, Peer-Timo Bremer, Erik W Draeger, Bernd Hamann, Katherine E Isaacs, Aaditya G Landge, Joshua A Levine, Valerio Pascucci, et al. Mapping applications with collectives over sub-communicators on torus networks. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1--11. IEEE, 2012.Google Scholar
- Abhinav Bhatele, Nikhil Jain, Yarden Livnat, Valerio Pascucci, and Peer-Timo Bremer. Analyzing network health and congestion in dragonfly-based supercomputers. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 93--102, 2016. Google ScholarCross Ref
- W. Michael Brown, Peng Wang, Steven J. Plimpton, and Arnold N. Tharrington. Implementing molecular dynamics on hybrid high performance computers âĂŞ short range forces. Computer Physics Communications, 182(4):898--911, 2011. Google ScholarCross Ref
- F. Chen, M. Flatken, A. Basermann, A. Gerndt, J. Hetheringthon, T. KrÃijger, G. Matura, and R. W. Nash. Enabling In Situ Pre- and Post-processing for Exascale Hemodynamic Simulations - A Co-design Study with the Sparse Geometry Lattice-Boltzmann Code HemeLB. In 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, pages 662--668, Nov 2012.Google Scholar
- Luiz DeRose, Bill Homer, and Dean Johnson. Detecting Application Load Imbalance on High End Massively Parallel Systems. In Proceedings of the 13th International Euro-Par Conference on Parallel Processing, Euro-Par'07, pages 150--159. Springer-Verlag, 2007. Google ScholarCross Ref
- Ciprian Docan, Manish Parashar, and Scott Klasky. DataSpaces: An Interaction and Coordination Framework for Coupled Simulation Workflows. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, HPDC '10, pages 25--36, 2010. Google ScholarDigital Library
- M. Dorier, G. Antoniu, F. Cappello, M. Snir, and L. Orf. Damaris: How to Efficiently Leverage Multicore Parallelism to Achieve Scalable, Jitter-free I/O. In 2012 IEEE International Conference on Cluster Computing, pages 155--163, Sept 2012. Google ScholarDigital Library
- Matthieu Dorier, Matthieu Dreher, Tom Peterka, Justin M. Wozniak, Gabriel Antoniu, and Bruno Raffin. Lessons learned from building in situ coupling frameworks. In Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, ISAV2015, pages 19--24, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- M. Dreher and B. Raffin. A Flexible Framework for Asynchronous in Situ and in Transit Analytics for Scientific Simulations. In Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on, pages 277--286, May 2014. Google ScholarDigital Library
- Megan Gilge. IBM System Blue Gene Solution: Blue Gene/Q Application Development. IBM Redbooks, June 2013.Google Scholar
- Salman Habib and et al. The Universe at Extreme Scale: Multi-petaflop Sky Simulation on the BG/Q. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012. Google ScholarDigital Library
- R.A. Haring, M. Ohmacht, T.W. Fox, M.K. Gschwind, D.L. Satterfield, K. Sugavanam, P.W. Coteus, P. Heidelberger, M.A. Blumrich, R.W. Wisniewski, A. Gara, G.L.-T. Chiu, P.A. Boyle, N.H. Chist, and Changhoan Kim. The IBM Blue Gene/Q Compute Chip. Micro, IEEE, 32(2):48--60, 2012.Google ScholarDigital Library
- Torsten Hoefler, Emmanuel Jeannot, and Guillaume Mercier. An Overview of Process Mapping Techniques and Algorithms in High-Performance Computing. In High Performance Computing on Complex Environments, pages 75--94. Wiley, 2014. Google ScholarCross Ref
- Tsuyoshi Ichimura, Kohei Fujita, Seizo Tanaka, Muneo Hori, Maddegedara Lalith, Yoshihisa Shizawa, and Hiroshi Kobayashi. Physics-based Urban Earthquake Simulation Enhanced by 10.7 BlnDOF × 30 K Time-step Unstructured FE Nonlinear Seismic Wave Simulation. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2014. Google ScholarDigital Library
- J. Jeffers, J. Reinders, and A. Sodani. Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition. Elsevier Science, 2016.Google ScholarDigital Library
- Geir Johansen. Managing Cray XT MPI Runtime Environment Variables to Optimize and Scale Applications. In Proceedings of the 2008 Cray User Group meeting, CUG '08, 2008.Google Scholar
- Peter Johnsen, Mark Straka, Melvyn Shapiro, Alan Norton, and Thomas Galarneau. Petascale WRF Simulation of Hurricane Sandy Deployment of NCSA's Cray XE6 Blue Waters. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2013. Google ScholarDigital Library
- Gary Lakner and et al. IBM System Blue Gene Solution: Performance Analysis Tools. IBM Redbooks, November 2008.Google Scholar
- Qing Liu, Jeremy Logan, Yuan Tian, Hasan Abbasi, Norbert Podhorszki, Jong Youl Choi, Scott Klasky, Roselyne Tchoua, Jay Lofstead, Ron Oldfield, Manish Parashar, Nagiza Samatova, Karsten Schwan, Arie Shoshani, Matthew Wolf, Kesheng Wu, and Weikuan Yu. Hello adios: the challenges and lessons of developing leadership class i/o frameworks. Concurrency and Computation: Practice and Experience, 26(7):1453--1473, 2014. Google ScholarDigital Library
- Preeti Malakar, Thomas George, Sameer Kumar, Rashmi Mittal, Vijay Natarajan, Yogish Sabharwal, Vaibhav Saxena, and Sathish S. Vadhiyar. A Divide and Conquer Strategy for Scaling Weather Simulations with Multiple Regions of Interest. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012. Google ScholarDigital Library
- Preeti Malakar, Venkatram Vishwanath, Christopher Knight, Todd Munson, and Michael E. Papka. Optimal execution of co-analysis for large-scale molecular dynamics simulations. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016. Google ScholarCross Ref
- Preeti Malakar, Venkatram Vishwanath, Todd Munson, Christopher Knight, Mark Hereld, Sven Leyffer, and Michael E. Papka. Optimal scheduling of in-situ analysis for large-scale scientific simulations. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2015. Google ScholarDigital Library
- Patrick OâĂ ŹLeary, James Ahrens, SÃl'bastien Jourdain, Scott Wittenburg, David H. Rogers, and Mark Petersen. Cinema image-based in situ analysis and visualization of MPAS-ocean simulations. Parallel Computing, 55:43--48, 2016. Visualization and Data Analytics for Scientific Discovery. Google ScholarDigital Library
- T. Peterka, J. Kwan, A. Pope, H. Finkel, K. Heitmann, S. Habib, Jingyuan Wang, and G. Zagaris. Meshing the universe: Integrating analysis in cosmological simulations. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 186--195, Nov 2012.Google Scholar
- Steve Plimpton. Fast parallel algorithms for short-range molecular dynamics. Journal of Computational Physics, 117(1):1--19, 1995. Google ScholarDigital Library
- Amanda Randles, Erik W. Draeger, Tomas Oppelstrup, Liam Krauss, and John A. Gunnels. Massively Parallel Models of the Human Circulatory System. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '15, 2015. Google ScholarDigital Library
- C. Seshadhri, Ali Pinar, David Thompson, and JanineC. Bennett. Sublinear algorithms for extreme-scale data analysis. In Janine Bennett, Fabien Vivodtzev, and Valerio Pascucci, editors, Topological and Statistical Methods for Complex Data, Mathematics and Visualization, pages 39--54. Springer Berlin Heidelberg, 2015.Google Scholar
- Christopher Sewell, Katrin Heitmann, Hal Finkel, George Zagaris, Suzanne T. Parete-Koon, Patricia K. Fasel, Adrian Pope, Nicholas Frontiere, Li-ta Lo, Bronson Messer, Salman Habib, and James Ahrens. Large-scale compute-intensive analysis via a combined in-situ and co-scheduling workflow approach. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '15, 2015. Google ScholarDigital Library
- Sameer S. Shende and Allen D. Malony. The Tau Parallel Performance System. International Journal of High Performance Computing Applications, 20(2):287--311, 2006. Google ScholarDigital Library
- A. Sodani, R. Gramunt, J. Corbal, H. S. Kim, K. Vinod, S. Chinthamani, S. Hutsell, R. Agarwal, and Y. C. Liu. Knights Landing: Second-Generation Intel Xeon Phi Product. IEEE Micro, 36(2):34--46, 2016. Google ScholarDigital Library
- DOE ASCAC Data Subcommittee. Synergistic Challenges in Data-Intensive Science and Exascale Computing. Technical report, March 2013.Google Scholar
- Brad Whitlock, Jean M. Favre, and Jeremy S. Meredith. Parallel in Situ Coupling of Simulation with a Fully Featured Visualization System. In Proceedings of the 11th Eurographics Conference on Parallel Graphics and Visualization, EGPGV '11, pages 101--109, Aire-la-Ville, Switzerland, Switzerland, 2011. Eurographics Association.Google ScholarDigital Library
- Boyu Zhang, Trilce Estrada, Pietro Cicotti, and Michela Taufer. Enabling In-Situ Data Analysis for Large Protein-Folding Trajectory Datasets. In Proceedings of the 2014 IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS '14, pages 221--230, 2014. Google ScholarDigital Library
- Fang Zheng, H. Abbasi, C. Docan, J. Lofstead, Qing Liu, S. Klasky, M. Parashar, N. Podhorszki, K. Schwan, and M. Wolf. PreDatA -- preparatory data analytics on peta-scale machines. In Parallel Distributed Processing (IPDPS), 2010 IEEE International Symposium on, April 2010.Google ScholarCross Ref
- Fang Zheng, Hongbo Zou, G. Eisenhauer, K. Schwan, M. Wolf, J. Dayal, Tuan-Anh Nguyen, Jianting Cao, H. Abbasi, S. Klasky, N. Podhorszki, and Hongfeng Yu. FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics. In Parallel Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, pages 320--331, May 2013.Google ScholarDigital Library
Recommendations
MIC acceleration of short-range molecular dynamics simulations
COSMIC '13: Proceedings of the First International Workshop on Code OptimiSation for MultI and many CoresHeterogeneous systems containing accelerators such as GPUs or co-processors such as Intel MIC are becoming more prevalent due to their ability of exploiting large-scale parallelism in applications. In this paper, we have developed a hierarchical ...
Towards Large-Scale Molecular Dynamics Simulations on Graphics Processors
BICoB '09: Proceedings of the 1st International Conference on Bioinformatics and Computational BiologyAtomistic molecular dynamics (MD) simulations are a vital tool in chemical research, as they are able to provide a view of chemical systems and processes that is not obtainable through experiment. However, large-scale MD simulations require access to ...
Increasing Molecular Dynamics Simulations Throughput by Virtualizing Remote GPUs with rCUDA
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel ProcessingThe full-understanding of the dynamics of molecular systems at the atomic scale is of great relevance in the fields of chemistry physics, materials science and drug discovery just to name a few. Molecular dynamics (MD) is a widely used computer tool for ...
Comments