skip to main content
10.1145/1995896.1995958acmconferencesArticle/Chapter ViewAbstractPublication PagesicsConference Proceedingsconference-collections
poster

SRC: enabling petascale data analysis for scientific applications through data reorganization

Published: 31 May 2011 Publication History

Abstract

This work proposes a novel data reorganization strategy to enable petascale data analysis for large-scale scientific applications running on high-end leadership computers. This strategy achieves optimal data layout for scientific applications by: 1) using a Space Filling Curve to reorganize the data chunks of multidimensional datasets on large storage systems, thus achieving a consistent and balanced read performance for any access pattern; 2) harmonizing the optimal chunk size with underlying file system for more efficient data access. Experimental results demonstrate a maximum of 37 times speedup for S3D simulation on the Jaguar supercomputer.

References

[1]
J. H. Chen et al. Terascale direct numerical simulations of turbulent combustion using S3D. Comp. Sci. & Disc., 2(1):015001 (31pp), 2009.
[2]
D. Hilbert. Ueber die stetige abbildung einer line auf ein flächenstück. Math. Ann., 38:459--460, 1891.
[3]
Oak Ridge National Laboratories. http://www.nccs.gov/user-support/center-projects/adios/.

Cited By

View all
  • (2012)SMART-IOProceedings of the 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems10.1109/MASCOTS.2012.30(181-188)Online publication date: 7-Aug-2012

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICS '11: Proceedings of the international conference on Supercomputing
May 2011
398 pages
ISBN:9781450301022
DOI:10.1145/1995896

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. data layout
  2. i/o
  3. space filling curve

Qualifiers

  • Poster

Conference

ICS '11
Sponsor:
ICS '11: International Conference on Supercomputing
May 31 - June 4, 2011
Arizona, Tucson, USA

Acceptance Rates

Overall Acceptance Rate 629 of 2,180 submissions, 29%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2012)SMART-IOProceedings of the 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems10.1109/MASCOTS.2012.30(181-188)Online publication date: 7-Aug-2012

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media