research-article

Integrated system models for reliable petascale storage systems

Author:

PDSW '07: Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07

Pages 12 - 16

https://doi.org/10.1145/1374596.1374600

Published: 11 November 2007 Publication History

Get Access

Abstract

The big challenges facing petascale storage systems are not actually those of performanc or traditional fault tolerance schemes, but instead the challenges stem from the inherent complexity of large distributed systems. The best way to address these problems is to build a system model into the system itself, and have the system continuously monitor its current state as compared with the desired state according to the model. Change is applied to the model first, and then the system strives to achieve the new model. The hard part is defining a good model, building good support for introspection, and exploiting these features in the system design. This paper describes this problem in more detail, explains some of the techniques we have used in building the Panasas distributed storage system, and concludes with a summary of the open problems with building petascale storage systems.

References

[1]

Gregory R. Ganger, John D. Strunk, Andrew J. Klosterman. Self-* Storage: Brick-based Storage with Automated Administration. Published as Carnegie Mellon University Technical Report, CMU-CS-03-178, August 2003.

Crossref

Google Scholar

[2]

Jeffrey O. Kephart, David M. Chess. "The Vision of Autonomic Computing." Computer, Vol. 36 No. 1, 41--50, 2003.

Digital Library

Google Scholar

[3]

L. Lamport. "The Part-Time Parliament." ACM Transactions on Computer Systems, Vol. 16 No. 2, 133--169, 1998

Digital Library

Google Scholar

[4]

D. Nagle, D. Serenyi, and A. Matthews. The Panasas ActiveScale storage cluster. Delivering scalable high bandwidth storage. In Proc. of the 2004 ACM/IEEE Conf. on Supercomputing, Nov. 2004.

Digital Library

Google Scholar

[5]

Oppenheimer, D., Archana Ganapathi, and David A. Patterson. Why do Internet services fail, and what can be done about it? 4th USENIX Symposium on Internet Technologies and Systems (USITS '03), March 2003.

Digital Library

Google Scholar

[6]

van Renesse, R., Birman, K. and Vogels, W. "Astrolabe: A Robust and Scalable Technology for Distributed System Monitoring, Management, and Data Mining." ACM Transactions on Computer Systems, 21(2):164--206, May 2003

Digital Library

Google Scholar

[7]

S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7^th Symposium on Operating Systems Design and Implementation (OSDI), Seattle, WA, Nov. 2006. USENIX.

Digital Library

Google Scholar

[8]

B. Welch, Z. Abbasi, G. Gibson, B. Mueller, M. Unangst, J. Zelenka, B. Zhou. Scalable Performance of the Panasas Distributed File System, Proceedings of FAST08, January 2008, San Jose, CA. (to appear)

Digital Library

Google Scholar

[9]

Theodore M. Wong, Richard A. Golding, Joseph S. Glider, Elizabeth Borowsky, Ralph A. Becker-Szendy, Claudio Fleiner, Deepak R. Kenchammana-Hosekote, Omer A. Zaki. Kybos: self-management for distributed brick-base storage. Research report RJ 10356, IBM Almaden Research Center, 26 August 2005

Google Scholar

Cited By

View all

Patil SGibson GGanger GWilkes J(2011)Scale and concurrency of GIGA+Proceedings of the 9th USENIX conference on File and stroage technologies10.5555/1960475.1960488(13-13)Online publication date: 15-Feb-2011
https://dl.acm.org/doi/10.5555/1960475.1960488
Ross RChoudhary AGibson GLiao W(2010)Parallel Data Storage and AccessScientific Data Management10.1201/9781420069815-c2Online publication date: 6-May-2010
https://doi.org/10.1201/9781420069815-c2
Welch BUnangst MAbbasi ZGibson GMueller BSmall JZelenka JZhou B(2008)Scalable performance of the Panasas parallel file systemProceedings of the 6th USENIX Conference on File and Storage Technologies10.5555/1364813.1364815(1-17)Online publication date: 26-Feb-2008
https://dl.acm.org/doi/10.5555/1364813.1364815

Index Terms

Integrated system models for reliable petascale storage systems
1. General and reference
  1. Cross-computing tools and techniques
    1. Reliability
2. Software and its engineering
  1. Software creation and management
    1. Designing software
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management
    2. Extra-functional properties
      1. Software reliability

Recommendations

Data driven workflow planning in cluster management systems
HPDC '07: Proceedings of the 16th international symposium on High performance distributed computing

Traditional scientific computing has been associated with harnessing computation cycles within and across clusters of machines. In recent years, scientific applications have become increasingly data-intensive. This is especially true in the fields of ...
Middleware support for many-task computing

Many-task computing aims to bridge the gap between two computing paradigms, high throughput computing and high performance computing. Many-task computing denotes high-performance computations comprising multiple distinct activities, coupled via file ...
An analysis of computational workloads for the ORNL Jaguar system
ICS '12: Proceedings of the 26th ACM international conference on Supercomputing

This study presents an analysis of science application workloads for the Jaguar Cray XT5 system during its tenure as a 2.3 petaflop supercomputer at Oak Ridge National Laboratory. Jaguar was the first petascale system to be deployed for open science and ...

Comments

Information & Contributors

Information

Published In

PDSW '07: Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07

November 2007

72 pages

ISBN:9781595938992

DOI:10.1145/1374596

Conference Chair:
Garth A. Gibson
Carnegie Mellon University and Panasas Inc.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SC '07

Sponsor:

SIGARCH

SC '07: International Conference for High Performance Computing, Networking, Storage and Analysis

November 11, 2007

Nevada, Reno

Acceptance Rates

Overall Acceptance Rate 17 of 41 submissions, 41%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
142
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Patil SGibson GGanger GWilkes J(2011)Scale and concurrency of GIGA+Proceedings of the 9th USENIX conference on File and stroage technologies10.5555/1960475.1960488(13-13)Online publication date: 15-Feb-2011
https://dl.acm.org/doi/10.5555/1960475.1960488
Ross RChoudhary AGibson GLiao W(2010)Parallel Data Storage and AccessScientific Data Management10.1201/9781420069815-c2Online publication date: 6-May-2010
https://doi.org/10.1201/9781420069815-c2
Welch BUnangst MAbbasi ZGibson GMueller BSmall JZelenka JZhou B(2008)Scalable performance of the Panasas parallel file systemProceedings of the 6th USENIX Conference on File and Storage Technologies10.5555/1364813.1364815(1-17)Online publication date: 26-Feb-2008
https://dl.acm.org/doi/10.5555/1364813.1364815

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Data driven workflow planning in cluster management systems

Middleware support for many-task computing

An analysis of computational workloads for the ORNL Jaguar system

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations