Abstract
In situ analysis has emerged as a dominant paradigm for performing scalable visual analysis of extreme-scale computational simulation data. Compared to the traditional post hoc analysis pipeline where data is first stored into disks and then analyzed offline, in situ analysis processes data at the time its generation in the supercomputers so that the slow and expensive disk I/O is minimized. In this work, we present a new in situ visual analysis pipeline for the extreme-scale multiphase flow simulation MFiX-Exa and demonstrate how the pipeline can be used to process large particle fields in situ and produce informative visualizations of the data features. We deploy our analysis pipeline on Oak Ridge’s Summit supercomputer to study its in situ applicability and usefulness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Summit supercomputer. https://docs.olcf.ornl.gov/systems/summit_user_guide.html. Accessed 24 May 2022
Ahern, S., Shoshani, A., Ma, K., Choudhary, A.: Scientific discovery at the exascale. Report from the DOE ASCR 2011 Workshop on Exascale Data Management. Analysis, and Visualization, February 2011
Ahrens, J., Jourdain, S., OLeary, P., Patchett, J., Rogers, D.H., Petersen, M.: An image-based approach to extreme scale in situ visualization and analysis. In: SC14: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 424–434 (2014). https://doi.org/10.1109/SC.2014.40
AMReX: A software framework for massively parallel, block-structured adaptive mesh refinement (AMR) applications (2021). https://amrex-codes.github.io/amrex/index.html. Accessed 7 Apr 2021
Atzori, M., et al.: In-situ visualization of large-scale turbulence simulations in nek5000 with paraview catalyst . https://doi.org/10.1007/s11227-021-03990-3
Bauer, A.C., et al.: In situ methods, infrastructures, and applications on high performance computing platforms. Comput. Graph. Forum 35(3), 577–597 (2016). https://doi.org/10.1111/cgf.12930
Biswas, A., Ahrens, J.P., Dutta, S., Musser, J.M., Almgren, A.S., Turton, T.L.: Feature analysis, tracking, and data reduction: an application to multiphase reactor simulation MFiX-Exa for In-Situ use case. Comput. Sci. Eng. 23(01), 75–82 (2021). https://doi.org/10.1109/MCSE.2020.3016927
Camata, J.J., Silva, V., Valduriez, P., Mattoso, M., Coutinho, A.L.: In situ visualization and data analysis for turbidity currents simulation. Comput. Geosci. 110, 23–31 (2018). https://doi.org/10.1016/j.cageo.2017.09.013
Childs, H.: Data exploration at the exascale. Supercomput. Front. Innov. 2(3) (2015). http://superfri.org/superfri/article/view/78
Childs, H., et al.: A terminology for in situ visualization and analysis systems. Int. J. High Perform. Comput. Appl. 34(6), 676–691 (2020). https://doi.org/10.1177/1094342020935991
Dutta, S., Chen, C., Heinlein, G., Shen, H.W., Chen, J.: In situ distribution guided analysis and visualization of transonic jet engine simulations. IEEE Trans. Vis. Comput. Graph. 23(1), 811–820 (2017)
Optimizing a new technology to reduce power plant carbon dioxide emissions (2022). https://www.exascaleproject.org/optimizing-a-new-technology-to-reduce-power-plant-carbon-dioxide-emissions/. Accessed 3 Feb 2022
Exascale Computing Project (2022). https://www.exascaleproject.org/. Accessed 12 Feb 2022
Fabian, N., et al.: The ParaView coprocessing library: a scalable, general purpose in situ visualization library. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 89–96 (2011). https://doi.org/10.1109/LDAV.2011.6092322
Haimes, R.: pv3: a distributed system for large-scale unsteady cfd visualization. In: AIAA paper, pp. 94–0321 (1994)
He, W., et al.: Insitunet: deep image synthesis for parameter space exploration of ensemble simulations. IEEE Trans. Vis. Comput. Graph. 26(1), 23–33 (2020). https://doi.org/10.1109/TVCG.2019.2934312
Larsen, M., et al.: The alpine in situ infrastructure: ascending from the ashes of strawman. In: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization, pp. 42–46. ISAV 2017, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3144769.3144778
Lofstead, J.F., Klasky, S., Schwan, K., Podhorszki, N., Jin, C.: Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS). In: Proceedings of the 6th International Workshop on Challenges of Large Applications in Distributed Environments, pp. 15–24. CLADE 2008, ACM (2008). https://doi.org/10.1145/1383529.1383533
Lukasczyk, J., et al.: Cinema darkroom: a deferred rendering framework for large-scale datasets. In: 2020 IEEE 10th Symposium on Large Data Analysis and Visualization (LDAV), pp. 37–41 (2020). https://doi.org/10.1109/LDAV51489.2020.00011
MFIX-Exa (2022). https://amrex-codes.github.io/MFIX-Exa/docs_html/. Accessed 3 Feb 2022
Musser, J., et al.: MFIX-Exa: a path toward exascale CFD-DEM simulations. Int. J. High Perform. Comput. Appl. (2021). https://doi.org/10.1177/10943420211009293
Peterka, T., Croubois, H., Li, N., Rangel, S., Cappello, F.: Self-Adaptive Density Estimation of Particle Data. SIAM J. Sci. Comput. 38(5), S646–S666 (2016). SISC Special Edition on CSE’15: Software and Big Data
Schroeder, W., Martin, K., Lorensen, B.: The Visualization Toolkit: An Object Oriented Approach to 3D Graphics, fourth edn. Kitware Inc. (2004). iSBN 1-930934-19-X
SENSEI: Scalable in situ analysis and visualization (2021). https://sensei-insitu.org/. Accessed 12 Feb 2022
Tikhonova, A., Correa, C., Ma, K.L.: Explorable images for visualizing volume data. In: 2010 IEEE Pacific Visualization Symposium (PacificVis), pp. 177–184 (2010). https://doi.org/10.1109/PACIFICVIS.2010.5429595
Vishwanath, V., Hereld, M., Papka, M.E.: Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 9–14 (2011). https://doi.org/10.1109/LDAV.2011.6092178
Whitlock, B., Favre, J.M., Meredith, J.S.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics Conference on Parallel Graphics and Visualization, pp. 101–109. EGPGV 2011, Eurographics Association (2011). https://doi.org/10.2312/EGPGV/EGPGV11/101-109
Woodring, J., Petersen, M., Schmei\(\beta \)er, A., Patchett, J., Ahrens, J., Hagen, H.: In situ eddy analysis in a high-resolution ocean climate model. IEEE Trans. Vis. Comput. Graph. 22(1), 857–866 (2016). https://doi.org/10.1109/TVCG.2015.2467411
Yi, H., Rasquin, M., Fang, J., Bolotnov, I.A.: In-situ visualization and computational steering for large-scale simulation of turbulent flows in complex geometries. In: 2014 IEEE International Conference on Big Data (Big Data), pp. 567–572 (2014). https://doi.org/10.1109/BigData.2014.7004275
Zhang, W., Myers, A., Gott, K., Almgren, A., Bell, J.: Amrex: block-structured adaptive mesh refinement for multiphysics applications. Int. J. High Perform. Comput. Appl. 35(6), 508–526 (2021). https://doi.org/10.1177/10943420211022811
Acknowledgements
The authors would like to thank the Department of Energy and Los Alamos National Laboratory for the funding and support in carrying out this research. This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. We thank our many ECP collaborators especially Jordan Musser, Ann Almgren, and Patrick O’Leary. This research is released under LA-UR-22-21278.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Dutta, S., Lipsa, D., Turton, T.L., Geveci, B., Ahrens, J. (2022). In Situ Analysis and Visualization of Extreme-Scale Particle Simulations. In: Anzt, H., Bienz, A., Luszczek, P., Baboulin, M. (eds) High Performance Computing. ISC High Performance 2022 International Workshops. ISC High Performance 2022. Lecture Notes in Computer Science, vol 13387. Springer, Cham. https://doi.org/10.1007/978-3-031-23220-6_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-23220-6_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23219-0
Online ISBN: 978-3-031-23220-6
eBook Packages: Computer ScienceComputer Science (R0)