skip to main content
10.1145/1645164.1645172acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

Plasma fusion code coupling using scalable I/O services and scientific workflows

Published: 16 November 2009 Publication History

Abstract

In order to understand the complex physics of mother nature, physicist often use many approximations to understand one area of physics and then write a simulation to reduce these equations to ones that can be solved on a computer. Different approximations lead to different equations that model different physics, which can often lead to a completely different simulation code. As computers become more powerful, scientists can either write one simulation that models all of the physics or they produce several codes each for different portions of the physics and then 'couple' these codes together. In this paper, we concentrate on the latter, where we look at our code coupling approach for modeling a full device fusion reactor. There are many approaches to code coupling. Our first approach was using Kepler workflows to loosely couple three codes via files (memory-to-disk-to-memory coupling). This paper describes our new approach moving towards using memory-to-memory data exchange to allow for a tighter coupling. Our approach focuses on a method which brings together scientific workflows along with staging I/O methods for code coupling. Staging methods use additional compute nodes to perform additional tasks such as data analysis, visualization, and NxM transfers for code coupling. In order to transparently allow application scientist to switch from memory to memory coupling to memory to disk to memory coupling, we have been developing a framework that can switch between these two I/O methods and then automate other workflow tasks. Our hybrid approach allows application scientist to easily switch between in-memory coupling and file-based coupling on-the-fly, which aids debugging these complex configurations.

References

[1]
C. S. Chang et al. "Toward a first-principles integrated simulation of tokamak edge plasmas", Journal of Physics: Conf. Ser. 125 012042 (7pp)
[2]
Z. Lin, S. Ethier, T. S. Hahm and W. M. Tang, "Size Scaling of Turbulent Transport in Magnetically Confined Plasmas", Phys. Rev. Lett. vol. 88, no. 19, p. 195004, Apr. 2002
[3]
C. S. Chang, S. Ku and H. Weitzner, "Numerical study of neoclassical plasma pedestal in a tokamak geometry", Phys. Plasmas, vol. 11, p. 2649, May 2004.
[4]
L. Zhang, M. Parashar, "A Dynamic Geometry-based Shared Space Interaction Framework for Parallel Scientific Applications", Proceedings of the 11th Annual International Conference on High Performance Computing (HiPC 2004), Bangalore, India, December 2004
[5]
R. Barreto, S. Klasky, N. Podhorszki, P. Mouallem and M. Vouk, "Collaboration Portal for Petascale Simulations". 2009 International Symposium on Collaborative Technologies and Systems, (CTS 2009), pp. 384--393, Baltimore, Maryland, USA, May 2009.
[6]
S. Klasky et al., "Adaptive IO System", Proceedings of the 50th Cray User Group meeting (CUG 2008), Helsinki, Finland, May 2008.
[7]
B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao and Y. Zhao, "Scientific Workflow Management and the Kepler System", Concurrency and Computation: Practice&Experience, vol. 18(10), p. 1039, August 2006.
[8]
L. A. Drummond, J. Demmel, C. R. Mechoso, H. Robinson and K. Sklower and J. A. Spahr. "A Data Broker for Distributed Computing Environments". In Lecture Notes in Computer Science. Volume 2073/2001, pp. 31--40, January 2001.
[9]
J. Cummings et al. "EFFIS: an End-to-end Framework for Fusion Integrated Simulation", submitted to PDP 2010, Pisa, Italy, February 2010
[10]
N. Podhorszki, B. Ludäscher, S. Klasky, "Workflow Automation for Processing Plasma Fusion Simulation Data", 2nd Workshop on Workflows in Support of Large-Scale Science (WORKS'07), June 25, 2007, Monterey, California, U.S.A.
[11]
J. Cummings et al. "Plasma edge kinetic-MHD modeling in tokamaks using Kepler workflow for code coupling, data management and visualization". Communications in Computational Physics, 4 (2008), pp. 675--702.
[12]
B. Ludäscher, I. Altintas, S. Bowers, J. Cummings, T. Critchlow, E. Deelman, D. D. Roure, J. Freire, C. Goble, M. Jones, S. Klasky, T. McPhillips, N. Podhorszki, C. Silva, I. Taylor, and M. Vouk. "Scientific Process Automation and Workflow Management". In A. Shoshani and D. Rotem, editors, Scientific Data Management: Challenges, Existing Technology, and Deployment, Computational Science Series, chapter 13. Chapman&Hall/CRC, 2009.
[13]
Ilkay Altintas et al, "Provenance in Kepler-based Scientific Workflow Systems," Poster # 41, at Microsoft eScience Workshop Friday Center, University of North Carolina, Chapel Hill, NC, October 13--15, 2007, pp. 82
[14]
P. Mouallem, M. Vouk, S. Klasky, N. Podhorszki and R. Barreto, "Tracking Files Using the Kepler Provenance Framework". 21st Intl. Conf. on Scientific and Statistical Database Management, SSDBM'09, LNCS 5566, pp. 273--282, New Orleans, LA, USA, June 2009
[15]
J. Lofstead, S. Klasky, M. Booth, H. Abbasi, F. Zheng, M. Wolf and K. Schwan, "Petascale IO using the Adaptable IO System", Proceedings of the 51st Cray User Group meeting (CUG 2009), Atlanta, GA, May 2009.
[16]
J. Lofstead, F. Zheng, S. Klasky and K. Schwan, "Adaptable, Metadata Rich IO Methods for Portable High Performance IO", Proceedings of the 23rd IEEE International Parallel&Distributed Processing Symposium (IPDPS 2009), Rome, Italy, May 2009.
[17]
H. Abbasi, J. Lofstead, F. Zheng, S. Klasky, K. Schwan and M. Wolf, "Extending I/O through High Performance Data Services", to appear at Cluster Computing 2009, New Orleans, LA, August 2009.
[18]
J. Lofstead, S. Klasky, K. Schwan, N. Podhorszki and C. Jin, "Flexible IO and Integration for Scientific Codes through the Adaptable IO System (ADIOS)", Proceedings of the 6th ACM/IEEE International Workshop on Challenges of Large Applications in Distributed Environments (CLADE 2008), Boston, MA, June 2008.
[19]
C. Docan, M. Parashar and S. Klasky, "DART: A Substrate for High Speed Asynchronous Data IO", In Proceedings of High Performance and Distributed Computing (HPDC'08), June 23--27, 2008, Boston, Massachusetts, USA.
[20]
W. Park, E. V. Belova, G. Y. Fu, X. Z. Tang, H. R. Strauss and L. E. Sugiyama, "Plasma simulation studies using multilevel physics models", Phys. Plasmas, vol. 6, p. 1796, May 1999
[21]
H. R. Wilson, P. B. Snyder, G. T. A. Huysmans and R. L. Miller, "Numerical studies of edge localized instabilities in tokamaks", Phys. Plasmas, vol. 9, p. 1277, April 2002.
[22]
C. Docan et al. "Experiments with Memory-to-Memory Coupling on Fusion Simulations", submitted for 5th Intl. Conf on e-Science, Oxford, UK, Dec. 2009

Cited By

View all
  • (2020)Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS47924.2020.00028(180-190)Online publication date: May-2020
  • (2017)A flexible I/O arbitration framework for netCDF‐based big data processing workflows on high‐end supercomputersConcurrency and Computation: Practice and Experience10.1002/cpe.416129:15Online publication date: 15-May-2017
  • (2016)Toward a General I/O Arbitration Framework for netCDF Based Big Data ProcessingProceedings of the 22nd International Conference on Euro-Par 2016: Parallel Processing - Volume 983310.1007/978-3-319-43659-3_22(293-305)Online publication date: 24-Aug-2016
  • Show More Cited By

Index Terms

  1. Plasma fusion code coupling using scalable I/O services and scientific workflows

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WORKS '09: Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science
    November 2009
    136 pages
    ISBN:9781605587172
    DOI:10.1145/1645164
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 November 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. code coupling
    2. parallel I/O
    3. plasma simulation
    4. workflow design
    5. workflow execution

    Qualifiers

    • Research-article

    Conference

    SC '09
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 30 of 54 submissions, 56%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 03 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2020)Understanding the Interplay between Hardware Errors and User Job Characteristics on the Titan Supercomputer2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS47924.2020.00028(180-190)Online publication date: May-2020
    • (2017)A flexible I/O arbitration framework for netCDF‐based big data processing workflows on high‐end supercomputersConcurrency and Computation: Practice and Experience10.1002/cpe.416129:15Online publication date: 15-May-2017
    • (2016)Toward a General I/O Arbitration Framework for netCDF Based Big Data ProcessingProceedings of the 22nd International Conference on Euro-Par 2016: Parallel Processing - Volume 983310.1007/978-3-319-43659-3_22(293-305)Online publication date: 24-Aug-2016
    • (2015)In-situ feature-based objects tracking for data-intensive scientific and enterprise analytics workflowsCluster Computing10.1007/s10586-014-0396-618:1(29-40)Online publication date: 1-Mar-2015
    • (2014)Enabling composite applications through an asynchronous shared memory interface2014 IEEE International Conference on Big Data (Big Data)10.1109/BigData.2014.7004236(219-224)Online publication date: Oct-2014
    • (2013)HobbesProceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers10.1145/2491661.2481427(1-8)Online publication date: 10-Jun-2013
    • (2013)Experiences applying data staging technology in unconventional waysProceedings of the 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing10.1109/CCGrid.2013.27(294-301)Online publication date: 13-May-2013
    • (2012)D2TProceedings of the 2012 IEEE International Conference on Cluster Computing10.1109/CLUSTER.2012.79(90-98)Online publication date: 24-Sep-2012
    • (2011)High end scientific codes with computational I/O pipelinesProceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities10.1145/2110205.2110210(23-28)Online publication date: 14-Nov-2011

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media