Skip to main content
SpringerLink
Log in

Practical prefetching techniques for multiprocessor file systems

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

Improvements in the processing speed of multiprocessors are outpacing improvements in the speed of disk hardware. Parallel disk I/O subsystems have been proposed as one way to close the gap between processor and disk speeds. In a previous paper we showed that prefetching and caching have thepotential to deliver the performance benefits of parallel file systems to parallel applications. In this paper we describe experiments withpractical prefetching policies that base decisions only on on-line reference history, and that can be implemented efficiently. We also test the ability of those policies across a range of architectural parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. M.G. Baker, J.H. Hartman, M.D. Kupfer, K.W. Shirriff, and J.K. Ousterhout, “Measurements of a distributed file system,” inProc. 13th ACM Symp. Operating Systems Principles, 1991, pp. 198–212.

  2. BBN Advanced Computers.Butterfly Products Overview, 1987.

  3. J.L. Bell, “A specialized data management system for parallel execution of particle physics codes,” inACM SIGMOD Conf., 1988, pp. 277–285.

  4. L.-F. Cabrera and D.D.E. Long, “Swift: Using distributed disk striping to provide high I/O data rates,” Technical Report CRL-91-46, UC Santa Cruz, 1991,Computing Systems. To appear.

  5. T.W. Crockett, “File concepts for parallel I/O,” inProc. Supercomputing '89, 1989, pp. 574–579.

  6. E. DeBenedictus and J.M. del Rosario, “nCUBE parallel I/O software,” inEleventh Ann. IEEE Int. Phoenix Conf. Computers Communications (IPCCC), April 1992, pp. 0117–0124.

  7. P.C. Dibble, “A parallel interleaved file system,” Ph.D. thesis, University of Rochester, 1990.

  8. P. Dibble, M. Scott, and C. Ellis, “Bridge: A high-performance file system for parallel processors,” inProc. 8th Int. Conf. Distributed Computer Systems, 1988, pp. 154–161.

  9. R. Floyd, “Short-term file reference patterns in a UNIX environment,” Technical Report 177, Dept. of Computer Science, Univ. of Rochester, 1986.

  10. R.A. Floyd, and C.S. Ellis, “Directory reference patterns in hierarchical file systems,”IEEE Trans. Knowledge Data Eng., vol. 1, pp. 238–247, 1989.

    Google Scholar 

  11. J.C. French, T.W. Pratt, and M. Das, “Performance measurement of a parallel input/output system for the Intel iPSC/2 hypercube,” inProc. 1991 ACM Sigmetrics Conf. Measurement and Modeling Computer Systems, 1991, pp. 178–187.

  12. “Intel beefs up its iPSC/2 supercomputer's I/O and memory capabilities,”Electronics, p. 24, Nov. 1988.

  13. iPSC/2 I/O facilities, Intel Corporation, 1988. Order number 280120-001.

  14. Paragon XP/S product overview, Intel Corporation, 1991.

  15. M.Y. Kim, “Synchronized disk interleaving,”IEEE Trans. Comput., vol. C-35, pp. 978–988, 1986.

    Google Scholar 

  16. D. Kotz, “Prefetching and caching techniques in file systems for MIMD multiprocessors,” Ph.D. thesis, Duke University, 1991. Available as technical report CS-1991-016.

  17. D. Kotz and C.S. Ellis, “Prefetching in file systems for MIMD multiprocessors,”IEEE Trans. Parallel Distributed Syst., vol. 1, pp. 218–230, 1990.

    Google Scholar 

  18. D. Kotz and C.S. Ellis, “Caching and writeback policies in parallel file systems,” in1991 IEEE Symposium Parallel and Distributed Processing, 1991, pp. 60–67, to appear inParallel Distributed Comput.

  19. E.L. Miller and R.H. Katz, “Input/output behavior of supercomputer applications,” inProc. Supercomputing '91, 1991, pp. 567–576.

  20. nCUBE Corporation, “nCUBE 2 supercomputers: Technical overview,” nCUBE brochure, 1990.

  21. J. Ousterhout, H.D. Costa, D. Harrison, J. Kunze, M. Kupfer, and J. Thompson, “A trace driven analysis of the UNIX 4.2 BSD file system,” inProc. 10th ACM Symp. Operating Systems Principles, 1985, pp. 15–24.

  22. J. Ousterhout and F. Douglis, “Beating the I/O bottleneck: A case for log-structured file systems,”ACM Oper. Syst. Rev., vol. 23, pp. 11–28, Jan. 1989.

    Google Scholar 

  23. D. Patterson, G. Gibson, and R. Katz, “A case for redundant arrays of inexpensive disks (RAID),” inACM SIGMOD Conf., 1989, pp. 109–116.

  24. P. Pierce, “A concurrent file system for a highly parallel mass storage system,” in4th Conf. Hypercube Concurrent Computers and Applications, 1989, pp. 155–160.

  25. T.W. Pratt, J.C. French, P.M. Dickens, and S.A. Janet, Jr., “A comparison of the architecture and performance of two parallel file systems,” in4th Conf. Hypercube Concurrent Computers and Applications, 1989, pp. 161–166.

  26. K. Salem and H. Garcia-Molina, “Disk striping,” inIEEE 1986 Conf. Data Engineering, 1986, pp. 336–342.

  27. A.J. Smith, “Sequential program prefetching in memory hierarchies,”IEEE Comput., vol. 11, pp. 7–21, Dec. 1978.

    Google Scholar 

  28. A.J. Smith, “Sequentiality and prefetching in database systems,”ACM Trans. Database Syst. vol. 3, pp. 223–247, 1978.

    Google Scholar 

  29. A.J. Smith, “Cache memories,”Comput. Surv. vol. 14, pp. 473–530, 1982.

    Google Scholar 

  30. A.J. Smith, “Disk cache-miss ratio analysis and design considerations,”ACM Trans. Comput. Syst. vol. 3, pp. 161–203. 1985.

    Google Scholar 

  31. A. Tucker and A. Gupta, “Process control and scheduling issues for multi-programmed sharedmemory multiprocessors,” inProc. 12th ACM Symp. Operating Systems Principles, 1989, pp. 159–166.

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, Dartmouth College, 03755-3551, Hanover, NH

    David Kotz

  2. Department of Computer Science, Duke University, 27708-0129, Durham, NC

    Carla Schlatter Ellis

Authors
  1. David Kotz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carla Schlatter Ellis
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kotz, D., Ellis, C.S. Practical prefetching techniques for multiprocessor file systems. Distrib Parallel Databases 1, 33–51 (1993). https://doi.org/10.1007/BF01277519

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01277519

Keywords

Search

Navigation