skip to main content
10.1145/2535571.2535593acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

A model for optimizing file access patterns using spatio-temporal parallelism

Published: 17 November 2013 Publication History

Abstract

For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. In this paper, we introduce a model that can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. Read times ultimately depend on how the file is stored and the access pattern used to read the file. The file access pattern will be dictated by the type of parallel decomposition used. We employ spatio-temporal parallelism, which combines both spatial and temporal parallelism, to provide greater flexibility to possible file access patterns. Using our model, we were able to configure the spatio-temporal parallelism to design optimized read access patterns that resulted in a speedup factor of approximately 400 over traditional file access patterns.

References

[1]
UV-CDAT Spatio-Temporal Parallel Processing Tools. http://uv-cdat.llnl.gov/presentations/PDF/ParaViewSTPWiki.pdf, 2013.
[2]
J. Biddiscombe, B. Geveci, K. Martin, K. Moreland, and D. Thompson. Time dependent processing in a parallel pipeline architecture. IEEE Transactions on Visualization and Computer Graphics, 13(6): 1376--1383, Nov. 2007.
[3]
D. Camp, H. Childs, A. Chourasia, C. Garth, and K. I. Joy. Evaluating the benefits of an extended memory hierarchy for parallel streamline algorithms. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 57--64. IEEE, 2011.
[4]
H. Childs, D. Pugmire, S. Ahern, B. Whitlock, M. Howison, G. H. Weber, E. W. Bethel, et al. Extreme scaling of production visualization software on diverse architectures. Computer Graphics and Applications, IEEE, 30(3): 22--31, 2010.
[5]
N. Fabian, K. Moreland, D. Thompson, A. C. Bauer, P. Marion, B. Gevecik, M. Rasquin, and K. E. Jansen. The paraview coprocessing library: A scalable, general purpose in situ visualization library. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 89--96. IEEE, 2011.
[6]
W. Kendall, J. Huang, T. Peterka, R. Latham, and R. Ross. Toward a general i/o layer for parallel-visualization applications. Computer Graphics and Applications, IEEE, 31(6): 6--10, 2011.
[7]
C. Michell, J. Ahrens, and J. Wang. Visio: Enabling interactive visualization of ultra-scale, time series data via high-bandwidth distributed i/o systems. pages 1--12. IEEE International Parallel and Distributed Processing Symposium, May 2011.
[8]
M. L. Norman and A. Snavely. Accelerating data-intensive science with gordon and dash. In Proceedings of the 2010 TeraGrid Conference, page 14. ACM, 2010.
[9]
T. Peterka, R. Ross, A. Gyulassy, V. Pascucci, W. Kendall, H.-W. Shen, T.-Y. Lee, and A. Chaudhuri. Scalable parallel building blocks for custom data analysis. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 105--112. IEEE, 2011.
[10]
Prabhat, O. Rbel, S. Byna, K. Wu, F. Li, M. Wehner, and W. Bethel. Teca: A parallel toolkit for extreme climate analysis. Procedia Computer Science, 9(0): 866--876, 2012. Proceedings of the International Conference on Computational Science, 2012.
[11]
V. Vishwanath, M. Hereld, and M. E. Papka. Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 9--14. IEEE, 2011.
[12]
B. Whitlock, J. M. Favre, and J. S. Meredith. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pages 101--109. Eurographics Association, 2011.
[13]
D. Williams, C. Doutriaux, J. Patchett, S. Williams, G. Shipman, R. Miller, C. Steed, H. Krishnan, C. Silva, A. Chaudhary, P. Bremer, D. Pugmire, W. Bethel, H. Childs, M. Prabhat, B. Geveci, A. Bauer, A. Pletzer, J. Poco, T. Ellqvist, E. Santos, G. Potter, B. Smith, T. Maxwell, D. Kindig, and D. Koop. The ultra-scale visualization climate data analysis tools (uv-cdat): Data analysis and visualization for geoscience data. Computer, PP(99): 1--1, 2013.
[14]
M. Woitaszek, J. M. Dennis, and T. R. Sines. Parallel high-resolution climate data analysis using swift. In Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers, MTAGS '11, pages 5--14, New York, NY, USA, 2011. ACM.
[15]
J. Woodring, S. Mniszewski, C. Brislawn, D. DeMarle, and J. Ahrens. Revisiting wavelet compression for large-scale climate data using jpeg 2000 and ensuring data precision. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 31--38. IEEE, 2011.
[16]
H. Yu and K.-L. Ma. A study of i/o methods for parallel visualization of large-scale data. Parallel Computing, 31(2): 167--183, 2005. Parallel Graphics and Visualization.
[17]
H. Yu, K.-L. Ma, and J. Welling. A parallel visualization pipeline for terascale earthquake simulations. In Proceedings of the 2004 ACM/IEEE conference on Supercomputing, SC '04, pages 49--, Washington, DC, USA, 2004. IEEE Computer Society.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UltraVis '13: Proceedings of the 8th International Workshop on Ultrascale Visualization
November 2013
56 pages
ISBN:9781450325004
DOI:10.1145/2535571
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. I/O
  2. data analysis
  3. modeling
  4. parallel techniques
  5. visualization

Qualifiers

  • Research-article

Funding Sources

Conference

SC13

Acceptance Rates

UltraVis '13 Paper Acceptance Rate 6 of 7 submissions, 86%;
Overall Acceptance Rate 6 of 7 submissions, 86%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 95
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media