Skip to main content
SpringerLink
Log in

‘Perfectly’ Scalable Data I/O

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This paper describes and analyzes a new architecture for file systems in which ‘metadata’, lock control, etc., are distributed among diverse resources. The basic data structure is a segment, viz. a logical group of files, folders, or other objects. The file system requires only one root, and can be non-hierarchical without a complete tree structure within segments. For ‘embarrassingly parallel’ data distributions, scalability is trivially perfect for all N,where N is the number of servers. Even for random file access, a new extreme statistical mechanics is used to show that data I/O is ‘perfectly’ scalable with probability 1, with degradation from perfect scaling that is small and bounded by f ln N/ ln (ln N). Here f is the fraction of data that is metadata. In contrast, earlier solutions degrade much faster, like Nf. No structural changes in classical metadata are required.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Article 23 July 2015

Juan Chabkinian & Thomas J. E. Schwarz SJ

References

  • Barker R., Massiglia P. (2001). Storage Area Networking Essentials; A Complete Guide to Understanding & Implementing SANs. Wiley.

  • Bender C.M., Orszag S.A. (1998). Advanced Mathematics Methods for Scientists and Engineers. Springer.

  • Berman, F., Fox, G., and Hey, A. J. G. (2003). Grid Computing: Making the Global Infrastructure a Reality. Wiley.

  • Buyya, R. (1999). High Performance Cluster Computing. Vol. 1 and 2. Prentice-Hall.

  • Farley, M. (2004). Storage Networking Fundamentals: An Introduction to Storage Devices, Applications, Management, and File Systems. Cisco Press.

  • Howard, J. H. (1988). An Overview of the Andrew File System. In Proceedings of the USENIX Winter Technical Conference.

  • Karniadakis, G. E., and Orszag, S. A. (1993). Nodes, modes, and flow codes. Phys. Today 34–42, March.

  • E. Levy A. Silberschatz (1990) ArticleTitleDistributed File Systems: Concepts and Examples ACM Computing Surveys. 22 IssueID4 321–373 Occurrence Handle10.1145/98163.98169

    Article  Google Scholar 

  • See http://encyclopedia.thefreedictionary.com.

  • Toigo, J. W., and Toigo, M. R. (2003). The Holy Grail of Storage Management. Prentice-Hall.

  • Patent, U. S. 6,782,389 (August 2004), related patents pending, and provisional patent applications, assigned to Ibrix, Inc.

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Yale University, P.O. Box 208283, New Haven, CT, 06520-8283

    Steven A. Orszag

  2. Department of Fluid Mechanics and Heat Transfer, Tel-Aviv University, Tel-Aviv, 69978, Israel

    Isaac Goldhirsch

  3. Ibrix, Inc., 300 Concord Road, Billerica, MA, 01854

    Sudhir Srinivasan

Authors
  1. Steven A. Orszag
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isaac Goldhirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sudhir Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Steven A. Orszag.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Orszag, S.A., Goldhirsch, I. & Srinivasan, S. ‘Perfectly’ Scalable Data I/O. J Sci Comput 24, 373–404 (2005). https://doi.org/10.1007/s10915-005-4811-2

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-005-4811-2

Keywords

Search

Navigation