Comparing the Efficiency of In Situ Visualization Paradigms at Scale

Kress, James M.; Larsen, Matthew; Choi, Jong Youl; Kim, Mark B.; Wolf, Matthew D.; Podhorszki, Norbert; Klasky, Scott A.; Childs, Hank; Pugmire, Dave

doi:10.1007/978-3-030-20656-7_6

Title: Comparing the Efficiency of In Situ Visualization Paradigms at Scale

Conference · Wed May 01 00:00:00 EDT 2019

DOI:https://doi.org/10.1007/978-3-030-20656-7_6· OSTI ID:1543212

^[1]; Larsen, Matthew ^[2];

^[1];

^[1]; Childs, Hank ^[3];

^[1]

ORNL
Lawrence Livermore National Laboratory (LLNL)
University of Oregon

This work compares the two major paradigms for doing in situ visualization: in-line, where the simulation and visualization share the same resources, and in-transit, where simulation and visualization are given dedicated resources. Our runs vary many parameters, including simulation cycle time, visualization frequency, and dedicated resources, to study how tradeoffs change over configuration. In particular, we consider simulations as large as 1,024 nodes (16,384 cores) and dedicated visualization resources with as many as 512 nodes (8,192 cores). We draw conclusions about when each paradigm is superior, such as in-line being superior when the simulation cycle time is very fast. Surprisingly, we also find that in-transit can minimize the total resources consumed for some configurations, since it can cause the visualization routines to require fewer overall resources when they run at lower concurrency. For example, one of our scenarios finds that allocating 25% more resources for visualization allows the simulation to run 61% faster than its in-line comparator. Finally, we explore various models for quantifying the cost for each paradigm, and consider transition points when one paradigm is superior to the other. Our contributions inform design decisions for simulation scientists when performing in situ visualization.

View Conference

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1543212

Resource Relation:: Journal Volume: 11501; Conference: ISC High Performance - Frankfurt, , Germany - 6/16/2019 8:00:00 AM-6/20/2019 8:00:00 AM

Country of Publication:: United States

Language:: English

References (18)

Characterizing and Modeling Power and Energy for Extreme-Scale In-Situ Visualization Adhinarayanan, Vignesh; Feng, Wu-chun; Rogers, David 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2017.113	conference	May 2017
ParaView: An End-User Tool for Large-Data Visualization Ahrens, James; Geveci, Berk; Law, Charles Visualization Handbook, p. 717-731 https://doi.org/10.1016/B978-012387582-2/50038-1	book	January 2005
The SENSEI Generic In Situ Interface Ayachit, Utkarsh; Whitlock, Brad; Wolf, Matthew 2016 Second Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV) https://doi.org/10.1109/ISAV.2016.013	conference	November 2016
ParaView Catalyst: Enabling In Situ Data Analysis and Visualization Ayachit, Utkarsh; Bauer, Andrew; Geveci, Berk Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV2015 https://doi.org/10.1145/2828612.2828624	conference	January 2015
Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures Ayachit, Utkarsh; Bauer, Andrew; Duque, Earl P. N. SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.78	conference	November 2016
Combining in-situ and in-transit processing to enable extreme-scale scientific analysis Bennett, Janine C.; Abbasi, Hasan; Bremer, Peer-Timo 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2012.31	conference	November 2012
Extreme Scaling of Production Visualization Software on Diverse Architectures Childs, Hank; Pugmire, David; Ahern, Sean IEEE Computer Graphics and Applications, Vol. 30, Issue 3 https://doi.org/10.1109/MCG.2010.51	journal	May 2010
Flexpath: Type-Based Publish/Subscribe System for Large-Scale Science Analytics Dayal, Jai; Bratcher, Drew; Eisenhauer, Greg 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) https://doi.org/10.1109/CCGrid.2014.104	conference	May 2014
DataSpaces: an interaction and coordination framework for coupled simulation workflows Docan, Ciprian; Parashar, Manish; Klasky, Scott Cluster Computing, Vol. 15, Issue 2 https://doi.org/10.1007/s10586-011-0162-y	journal	February 2011
Exploring power behaviors and trade-offs of in-situ data analytics Gamell, Marc; McCormick, Patrick; Pakin, Scott Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503303	conference	January 2013
Loosely Coupled In Situ Visualization: A Perspective on Why It's Here to Stay Kress, James; Klasky, Scott; Podhorszki, Norbert Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV2015 https://doi.org/10.1145/2828612.2828623	conference	January 2015
The ALPINE In Situ Infrastructure: Ascending from the Ashes of Strawman Larsen, Matthew; Ahrens, James; Ayachit, Utkarsh Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization - ISAV'17 https://doi.org/10.1145/3144769.3144778	conference	January 2017
Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks: HELLO ADIOS Liu, Qing; Logan, Jeremy; Tian, Yuan Concurrency and Computation: Practice and Experience, Vol. 26, Issue 7 https://doi.org/10.1002/cpe.3125	journal	August 2013
VTK-m: Accelerating the Visualization Toolkit for Massively Threaded Architectures Moreland, Kenneth; Sewell, Christopher; Usher, William IEEE Computer Graphics and Applications, Vol. 36, Issue 3 https://doi.org/10.1109/MCG.2016.48	journal	May 2016
Evaluation of methods to integrate analysis into a large-scale shock shock physics code Oldfield, Ron A.; Moreland, Kenneth; Fabian, Nathan Proceedings of the 28th ACM international conference on Supercomputing - ICS '14 https://doi.org/10.1145/2597652.2597668	conference	January 2014
SCIRun: a scientific programming environment for computational steering Parker, Steven G.; Johnson, Christopher R. Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '95 https://doi.org/10.1145/224170.224354	conference	January 1995
Evaluation of In-Situ Analysis Strategies at Scale for Power Efficiency and Scalability Rodero, Ivan; Parashar, Manish; Landge, Aaditya G. 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) https://doi.org/10.1109/CCGrid.2016.95	conference	May 2016
In-memory staging and data-centric task placement for coupled scientific simulation workflows: In-memory staging and data-centric task placement for coupled scientific simulation workflows Zhang, Fan; Jin, Tong; Sun, Qian Concurrency and Computation: Practice and Experience, Vol. 29, Issue 12 https://doi.org/10.1002/cpe.4147	journal	April 2017

Similar Records

Adaptive elasticity policies for staging-based in situ visualization

Journal Article · Wed Dec 21 00:00:00 EST 2022 · Future Generations Computer Systems · OSTI ID:1543212

Wang, Zhe; Dorier, Matthieu; Subedi, Pradeep; +2 more

A lightweight method for evaluating in situ workflow efficiency

Journal Article · Thu Dec 03 00:00:00 EST 2020 · Journal of Computational Science · OSTI ID:1543212

Do, Tu Mai Anh; Pottier, Loïc; Caíno-Lores, Silvina; +6 more

Center for Technology for Advanced Scientific Componet Software (TASCS)

Technical Report · Sun Oct 31 00:00:00 EDT 2010 · OSTI ID:1543212

Govindaraju, Madhusudhan

Title: Comparing the Efficiency of In Situ Visualization Paradigms at Scale

Citation Formats

References (18)

Similar Records

Related Subjects