Skip to main content
SpringerLink
Log in

Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets

  • Conference paper
  • First Online:
High Performance Computing (ISC High Performance 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11203))

Included in the following conference series:

  • 1295 Accesses

Abstract

This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ayachit, U., et al.: The SENSEI generic in situ interface. In: Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization, ISAV 2016, pp. 40–44. IEEE Press, Piscataway (2016). https://doi.org/10.1109/ISAV.2016.13

  2. Bauer, A.C., et al.: In situ methods, infrastructures, and applications on high performance computing platforms. Comput. Graph Forum 35(3), 577–597 (2016). https://doi.org/10.1111/cgf.12930. https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.12930

    Article  Google Scholar 

  3. Fareed, H., Singler, J.R.: Error Analysis of an Incremental POD Algorithm for PDE Simulation Data. ArXiv e-prints, March 2018

    Google Scholar 

  4. Hunter, J.D.: Matplotlib: a 2D graphics environment. Comput Sci Eng. 9(3), 90–95 (2007). https://doi.org/10.1109/MCSE.2007.55

    Article  Google Scholar 

  5. Research Institute for Information Technology, K.U.: Supercomputer System ITO. https://www.cc.kyushu-u.ac.jp/scp/eng/system/01_into.html. Accessed 15 May 2018

  6. Kress, J., Klasky, S., Podhorszki, N., Choi, J., Childs, H., Pugmire, D.: Loosely coupled in situ visualization: a perspective on why it’s here to stay. In: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, pp. 1-6. ISAV 2015. ACM, New York (2015). https://doi.org/10.1145/2828612.2828623, http://doi.acm.org/10.1145/2828612.2828623

  7. Nanri, T.: Proposal of interface for runtime memory manipulation of applications via PGAS-based communication library. In: Workshop on PGAS programming models: Experiences and Implementations, HPC Asia 2018, 31 January 2018

    Google Scholar 

  8. Ono, K., Kawashima, Y., Kawanabe, T.: Data centric framework for large-scale high-performance parallel computation. Procedia Comput. Sci. 29, 2336–2350 (2014). https://doi.org/10.1016/j.procs.2014.05.218, http://www.sciencedirect.com/science/article/pii/S1877050914003950. 2014 International Conference on Computational Science

    Article  Google Scholar 

  9. Uchida, T.: LES investigation of terrain-induced turbulence in complex terrain and economic effects of wind turbine control. Energies 11(6), 1530 (2018). https://doi.org/10.3390/en11061530

    Article  Google Scholar 

Download references

Acknowledgement

This research has used the computational resources of the K computer at RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. This work is partially supported by the “Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures” in Japan (Project ID: jh180060-NAH), and also by the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) as a social and scientific priority issue (Development of Innovative Design and Production Processes that Lead the Way for the Manufacturing Industry in the Near Future) to be tackled by using the post-K supercomputer.

Author information

Authors and Affiliations

  1. Research Institute for Information Technology, Kyushu University, Fukuoka, Japan

    Kenji Ono & Takeshi Nanri

  2. RIKEN Center for Computational Science, Kobe, Japan

    Kenji Ono, Jorji Nonaka, Yoshiyuki Morie, Tomohiro Kawanabe & Fumiyoshi Shoji

  3. Fujitsu Limited, Tokyo, Japan

    Hiroyuki Yoshikawa

Authors
  1. Kenji Ono
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jorji Nonaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiroyuki Yoshikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takeshi Nanri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yoshiyuki Morie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomohiro Kawanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fumiyoshi Shoji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenji Ono .

Editor information

Editors and Affiliations

  1. Tokyo Institute of Technology, Tokyo, Japan

    Rio Yokota

  2. University of Edinburgh, Edinburgh, UK

    Michèle Weiland

  3. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    John Shalf

  4. Swiss National Supercomputing Centre, Lugano, Switzerland

    Sadaf Alam

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ono, K. et al. (2018). Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. In: Yokota, R., Weiland, M., Shalf, J., Alam, S. (eds) High Performance Computing. ISC High Performance 2018. Lecture Notes in Computer Science(), vol 11203. Springer, Cham. https://doi.org/10.1007/978-3-030-02465-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02465-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02464-2

  • Online ISBN: 978-3-030-02465-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation