Magdalena Slawinska
ABSTRACT
There are many challenges in analyzing and visualizing data from current cutting-edge general relativistic astrophysics simulations. Many of the associated tasks are time-consuming with large performance degradation due to the magnitude and complexity of the data. The Adaptable I/O System (ADIOS) is a componentization of the I/O layer that has demonstrated remarkable I/O performance improvements on applications running on leadership class machines while also offering new in-memory "staging" operations for transforming data in-situ. We have been incorporating ADIOS staging technologies into our Maya numerical relativity code based on Cactus infrastructure and Carpet mesh refinement. Incorporating ADIOS into the Maya code is the first step toward enabling a more adaptable Maya workflow. We provide descriptions how we intend to leverage FlexPath (an ADIOS transport method) to provide a richer user experience in real-time visualization and interactive steering.
- Cactus 4.0 Reference Manual. http://einsteintoolkit.org/documentation/ReferenceManual/ReferenceManual.html, 2012.Google Scholar
- Cactus Framework website. http://cactuscode.org/, 2013.Google Scholar
- Carpet---Adaptive Mesh Refinement for the Cactus Framework website. http://www.carpetcode.org/, 2013.Google Scholar
- Einstein Toolkit website. http://einsteintoolkit.org/, 2013.Google Scholar
- ParaView CoProcessing Wiki. http://www.paraview.org/Wiki/CoProcessing, 2013.Google Scholar
- The HDF5 Group website. http://www.hdfgroup.org, 2013.Google Scholar
- VisIt website: Software that delivers parallel interactive visualization. https://wci.llnl.gov/codes/visit/home.html, 2013.Google Scholar
- Wolfram Research: Mathematica website. http://www.wolfram.com/mathematica/, 2013.Google Scholar
- XSEDE Remote Visualization Portal. https://portal.xsede.org/remote-visualization, 2013.Google Scholar
- R. Arnowitt, S. Deser, and C. W. Misner. Dynamical Structure and Definition of Energy in General Relativity. Phys. Rev., 116:1322--1330, Dec 1959.Google ScholarCross Ref
- J. C. Bennett, H. Abbasi, P.-T. Bremer, R. Grout, A. Gyulassy, T. Jin, S. Klasky, H. Kolla, M. Parashar, V. Pascucci, P. Pebay, D. Thompson, H. Yu, F. Zhang, and J. Chen. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 49:1--49:9, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press. Google ScholarDigital Library
- J. Biddiscombe, J. Soumagne, G. Oger, D. Guibert, and J.-G. Piccinali. Parallel Computational Steering for HPC Applications Using HDF5 Files in Distributed Shared Memory. IEEE Transactions on Visualization and Computer Graphics, 18(6):852--864, 2012. Google ScholarDigital Library
- J. D. Brunner, D. J. Jablonowski, B. Bliss, and R. B. Haber. VASE: the visualization and application steering environment. In Proceedings of the 1993 ACM/IEEE conference on Supercomputing, Supercomputing '93, pages 560--569, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- A. Cedilnik, B. Geveci, K. Moreland, J. Ahrens, and J. Favre. Remote large data visualization in the ParaView framework. In Proceedings of the 6th Eurographics conference on Parallel Graphics and Visualization, EG PGV'06, pages 163--170, Aire-la-Ville, Switzerland, Switzerland, 2006. Eurographics Association. Google ScholarDigital Library
- G. Eisenhauer, M. Wolf, H. Abbasi, and K. Schwan. Event-based systems: opportunities and challenges at exascale. In Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, DEBS '09, pages 2:1--2:10, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- B. Giacomazzo and L. Rezzolla. WhiskyMHD: a new numerical code for general relativistic magnetohydrodynamics. Classical and Quantum Gravity, 24(12):S235, 2007.Google ScholarCross Ref
- T. Goodale, G. Allen, G. Lanfermann, J. Masso, T. Radke, E. Seidel, and J. Shalf. The Cactus Framework and Toolkit: Design and Applications. Vector and Parallel Processing 2002, 5th International Conference, 2003. Google ScholarDigital Library
- W. Gu, G. Eisenhauer, E. Kraemer, K. Schwan, J. T. Stasko, J. Vetter, and N. Mallavarupu. Falcon: On-line Monitoring and Steering of Large-Scale Parallel Programs. In Proceedings of the 5th Symposium of the Frontiers of Massively Parallel Computing, McLean, VA,, pages 422--429, 1995. Google ScholarDigital Library
- S. Husa, I. Hinder, and C. Lechner. Kranc: A Mathematica application to generate numerical codes for tensorial evolution equations. Comput.Phys.Commun., 174:983--1004, 2006.Google ScholarCross Ref
- G. I. James, G. A. Geist, I. James, A. Kohl, and P. M. Papadopoulos. CUMULVS: Providing Fault-Tolerance, Visualization and Steering of Parallel Applications. International Journal of High Performance Computing Applications, 11:224--236, 1996.Google Scholar
- A. Kageyama and T. Yamada. An Approach to Exascale Visualization: Interactive Viewing of In-Situ Visualization. arXiv preprint arXiv:1301.4546, 2013.Google Scholar
- F. Löffler, J. Faber, E. Bentivegna, T. Bode, P. Diener, R. Haas, I. Hinder, B. C. Mundim, C. D. Ott, E. Schnetter, G. Allen, M. Campanelli, and P. Laguna. The Einstein Toolkit: A Community Computational Infrastructure for Relativistic Astrophysics. Classical and Quantum Gravity, 29(11):115001, 2012.Google ScholarCross Ref
- J. Lofstead, F. Zheng, S. Klasky, and K. Schwan. Adaptable, Metadata Rich IO Methods for Portable High Performance IO. In Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing, IPDPS '09, pages 1--10, Washington, DC, USA, 2009. IEEE Computer Society. Google ScholarDigital Library
- J. F. Lofstead, S. Klasky, K. Schwan, N. Podhorszki, and C. Jin. Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS). In Proceedings of the 6th international workshop on Challenges of large applications in distributed environments, CLADE '08, pages 15--24, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- S. Parker and C. Johnson. SCIRun: A Scientific Programming Environment for Computational Steering. In Supercomputing, 1995. Proceedings of the IEEE/ACM SC95 Conference, pages 52--52. Google ScholarDigital Library
- N. Podhorszki, Q. Liu, J. Logan, H. Abbasi, J. Choi, and S. Klasky. ADIOS 1.4.1 User's Manual. Office of Science, U.S. Department of Energy, Dec 2012.Google Scholar
- E. Schnetter, S. H. Hawley, and I. Hawke. Evolutions in 3D numerical relativity using fixed mesh refinement. Classical and Quantum Gravity, 21(6):1465, 2004.Google ScholarCross Ref
- The HDF Group. HDF5 User's Guide. HDF5 Release 1.8.10, Nov 2012. Chapter 3, Section 3.6.Google Scholar
- A. Tikhonova, H. Yu, C. D. Correa, J. H. Chen, and K.-L. Ma. A preview and exploratory technique for large-scale scientific simulations. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, EG PGV'11, pages 111--120, Aire-la-Ville, Switzerland, Switzerland, 2011. Eurographics Association. Google ScholarDigital Library
- T. Tu, H. Yu, J. Bielak, O. Ghattas, J. C. Lopez, K.-L. Ma, D. R. O'Hallaron, L. Ramirez-Guzman, N. Stone, R. Taborda-Rios, and J. Urbanic. Remote runtime steering of integrated terascale simulation and visualization. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing, SC '06, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- A. Tuchman, D. Jablonowski, and G. Cybenko. A System for Remote Data Visualization. Technical report, Center for Supercomputing Research and Development, University of Illinois at Urbana-Champaign, 1991.Google Scholar
- B. Whitlock, J. M. Favre, and J. S. Meredith. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, EG PGV'11, pages 101--109, Aire-la-Ville, Switzerland, Switzerland, 2011. Eurographics Association. Google ScholarDigital Library
- S. A. Williams, M. L. Sawley, and D. Cobut. Distributed run-time visualization and solution steering of parallel flow simulations. Comput. Phys., 12(5):493--502, Sept. 1998. Google ScholarDigital Library
- XSEDE website: Extreme Science and Engineering Discovery Environment. https://www.xsede.org/resources/overview, 2013.Google Scholar
- H. Yu, C. Wang, R. W. Grout, J. H. Chen, and K.-L. Ma. In situ visualization for large-scale combustion simulations. IEEE Comput. Graph. Appl., 30(3):45--57, May 2010. Google ScholarDigital Library
- F. Zheng, H. Zou, G. Eisenhauer, K. Schwan, M. Wolf, J. Dayal, T.-A. Nguyen, J. Cao, H. Abbasi, S. Klasky, N. Podhorszki, and H. Yu. FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics. 27th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2013), May 2013. Boston, MA. Google ScholarDigital Library
Index Terms
- A Maya use case: adaptable scientific workflows with ADIOS for general relativistic astrophysics
Recommendations
Highly Interactive, Steered Scientific Workflows on HPC Systems: Optimizing Design Solutions
High Performance ComputingAbstractScientific workflows are becoming increasingly important in high performance computing (HPC) settings, as the feasibility and appeal of many simultaneous heterogeneous tasks increases with increasing hardware capabilities. Currently no HPC-based ...
The Astrophysics Simulation Collaboratory: A Science Portal Enabling Community Software Development
Grid Portals, based on standard web technologies, are emerging as important and useful user interfaces to computational and data Grids. Grid Portals enable Virtual Organizations, comprised of distributed researchers to collaborate and access resources ...
A Case Study on Providing Accessibility-Focused In-Transit Architectures for Neural Network Simulation and Analysis
High Performance ComputingAbstractDue to the ever-increasing availability of high-performance computing infrastructure, developers can simulate increasingly complex models. However, the increased complexity comes with new challenges regarding data processing and visualization due ...
Comments