Skip to main content

Abstract

We sketch the reasons for the I/O bottleneck in parallel and distributed systems, pointing out that it can be viewed as a special case of a general bottleneck that arises at all levels of the memory hierarchy. We argue that because of its severity, the I/O bottleneck deserves systematic attention at all levels of system design. We then present a survey of the issues raised by the I/O bottleneck in five key areas of parallel and distributed systems: applications, algorithms, compilers, operating systems and architecture. Finally, we address some of the trends we observe emerging in new paradigms of parallel and distributed computing: the convergence of networking and I/O, I/O for massively distributed “global information systems” such as the World Wide Web, and I/O for mobile computing and wireless communications. These considerations suggest exciting new research directions in I/O for parallel and distributed systems in the years to come.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Acharya, M. Uysal, R. Bennett, A. Mendelson, M. Beynon, J. Hollingsworth, J. Saltz, and A. Sussman. Tuning the performance of I/O-intensive applications. In Proc. Workshop on I/O in Par. and Distrib. Sys. (IOPADS), 1996.

    Google Scholar 

  2. S. Acharya, R. Alonso, M. Franklin, and S. Zdonik. Broadcast disks: Data management for asymmetric communication environments. In Proc. SIGMOD, 1995.

    Google Scholar 

  3. A. Aggarwal and J. S. Vitter. The Input/Output complexity of sorting and related problems. Comm. ACM, pages 1116–1127, Sep. 1988.

    Google Scholar 

  4. J. Akella and D. P. Siewiorek. Modeling and measurement of the impact of Input/Output on system performance. In Proc. 18th Intl. Symp. Comp. Arch., pages 390–399, 1991.

    Google Scholar 

  5. D. P. Anderson, Y. Osawa, and R. Govindan. Real-time disk storage and retrieval of digital audio and video. ACM Trans. Comp. Sys., 1993.

    Google Scholar 

  6. M. Arrott and S. Latta. Perspectives on visualization. IEEE Spectrum, pages 61–65, Sep. 1992.

    Google Scholar 

  7. B. R. Badrinath and T. Imielinski. Data management issues in mobile computing. In Wireless Datacomm ’92, 1992.

    Google Scholar 

  8. F. Baskett. Keynote address. Intl. Symp. Shared Mem. Proc., Apr. 1991.

    Google Scholar 

  9. Sandra Johnson Baylor, Caroline Benveniste, and Yarsun Hsu. Performance evaluation of a massively parallel I/O subsystem. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  10. Sandra Johnson Baylor and C. Eric Wu. Parallel I/O workload characteristics using Vesta. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  11. M. Bender et al. Unix for nomads: Making Unix support mobile computing. In Proc. USENIX Symp. Mobile and Location-Independent Comp. Symp., pages 53–67, Aug. 1993.

    Google Scholar 

  12. H.Boral and D.J.DeWitt. Database machines: An idea whose time has passed? A critique of the future of database machines. In Third Intl. Workshop on Database Machines, pages 166–187, 1983.

    Google Scholar 

  13. Rajesh Bordawekar and Alok Choudhary. Issues in compiling I/O intensive problems. In Input/Outputin Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  14. D. Bradley and D. A. Reed. Performance of the Intel iPSC/2 input/output system. In Proc. Conf. on Hypercubes, Concurrent Comp, and Appl., pages 141–144, 1990.

    Google Scholar 

  15. J. C. Browne, Muhammad Azam, and Stephen Sobek. CODE: A unified approach to parallel programming. IEEE Software, page 11, July 1989.

    Google Scholar 

  16. J. C. Browne, A. Dale, C. Leung, and R. Jenevein. A parallel multi-stage I/O architecture with self-managing disk cache for database management applications. In Fourth Intl. Workshop on Database Machines. Springer-Verlag, 1985.

    Google Scholar 

  17. J. C. Browne, G. E. Onstott, P. L. Soffa, Ron Goering, S. Sivaramakrishnan, Harish Balan, and K. Somalwar. Design and evaluation of external memory architectures for multiprocessor computer systems: Second quarter report to IBM Yorktown Heights Research Lab. Technical report, Univ. Texas at Austin, Dept. of Comp. Sci., 1987. Available from J. C. Browne.

    Google Scholar 

  18. C. E. Catlett. Balancing resources. IEEE Spectrum, pages 48–55, Sep. 1992.

    Google Scholar 

  19. A. Chankhunthod, P. Danzig, C. Neerdaels, M. Schwartz, and K. Worrell. A hierarchical Internet object cache. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.

    Google Scholar 

  20. J. Child. Making every watt count. Computer Design, page 67, Dec. 1993.

    Google Scholar 

  21. H.-K. Jerry Chu. Zero-copy TCP in Solaris. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.

    Google Scholar 

  22. S. A. Coleman and R. W. Watson. New architectures to reduce I/O bottlenecks in high-performance systems. In Proc. 26th Hawaii Intl. Conf. Sys. Sci., page 5, 1993.

    Google Scholar 

  23. P. Corbett, D. Feitelson, S. Fineberg, Y. Hsu, B. Nitzberg, J.-P. Prost, M. Snir, B. Traversat, and P. Wong. Overview of the MPI-IO parallel I/O interface. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  24. T. C. Cormen and K. Bruhl. Don’t be too clever: Routing BMMC permutations on the MasPar MP-2. In Proc. Symp. Par. Algm. Arch. (SPAA), July 1995.

    Google Scholar 

  25. T. C. Cormen and A. Colvin. ViC*: A preprocessor for virtual-memory C*. Tech. Rep. PCS-TR94–243, Dept. of Comp. Sci., Dartmouth, Nov. 1994.

    Google Scholar 

  26. T. C. Cormen and D. Kotz. Integrating theory and practice in parallel file systems. Tech. Rep. PCS-TR93–188, Dept. of Comp. Sci., Dartmouth, Nov. 1994.

    Google Scholar 

  27. P. Crandall, R. A. Aydt, A. A. Chien, and D. A. Reed. Input/output characteristics of scalable parallel applications. In Proc. Supercomp. ’95, Dec. 1995.

    Google Scholar 

  28. D. E. Culler, R. M. Karp, D. A. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. Log P: Towards a realistic model of parallel computation. In Proc. Symp. Princ. Prac. of Par. Prog., May 1993.

    Google Scholar 

  29. R. Cypher, A. Ho, S. Konstantinidou, and P. Messina. Architectural requirements of parallel scientific applications with explicit communication. In Intl. Symp. Comp. Arch., 1993.

    Google Scholar 

  30. Juan Miguel del Rosario, R. Bordawekar, and Alok Chaudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. Workshop on I/O in Parallel Computer Systems, pages 56–70, 1993. Also in ACM SIGARCH Comp. Arch. News., Dec. 1993.

    Google Scholar 

  31. Juan Miguel del Rosario and Alok Choudhary. High-performance I/O for massively parallel computers: Problems and prospects. IEEE Computer, pages 59–68, Mar. 1994.

    Google Scholar 

  32. D. De Witt and J. Gray. Parallel database systems: The future of high performance database systems. CACM, 35:85–98, June 1992.

    Google Scholar 

  33. P. Druschel, M. B. Abbott, M. Pagels, and L. L. Peterson. Network subsystem design. IEEE Network, pages 8–17, July 1993.

    Google Scholar 

  34. D. Durand, R. Jain, and D. Tseytlin. Improving the performance of parallel I/O using distributed scheduling algorithms. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  35. K. Fall and J. Pasquale. Exploiting in-kernel data paths to improve I/O throughput and CPU availability. In Proc. Winter USENIX, pages 327–333, Jan. 1993.

    Google Scholar 

  36. High Performance Fortran Forum. High performance Fortran language specification. Sci. Prog., 2:1–170, 1993.

    Google Scholar 

  37. E. A. Fox, editor. Special Issue on digital multimedia systems. Comm. ACM, Apr. 1991.

    Google Scholar 

  38. G. Fox, S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, and C. Tseng. Fortran D language specifications. Technical Report COMP TR90–141, Rice Univ., 1990.

    Google Scholar 

  39. J. C. French, T. W. Pratt, and M. Das. Performance meaurement of a parallel Input/Output system for the Intel iPSC/2 hypercube. In Proc. SIGMETRICS, pages 178–187, 1991.

    Google Scholar 

  40. D. J. Gemmell, H. M. Vin, D. D. Kandlur, P. Venkat Rangan, and L. A. Rowe. Multimedia storage servers: A tutorial. IEEE Computer, pages 40–49, Mar. 1995.

    Google Scholar 

  41. J. Ghosh and B. Agarwal. Parallel I/O subsystems for hypercube multicomputers. In Proc. Intl. Par. Proc. Symp., pages 381–384, 1991.

    Google Scholar 

  42. Joydeep Ghosh, K. D. Goveas, and J. T. Draper. Performance evaluation of a parallel I/O subsystem for hypercube multicomputers. J. Par. Distrib. Comp., Jan/Feb. 1993.

    Google Scholar 

  43. G. A. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. PhD thesis, Univ. of Calif., Berkeley, Comp. Sci. Div, 1990. Also available as Tech. Rep. UCB/CSD 91/613.

    Google Scholar 

  44. G. A. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. The MIT Press, 1992.

    Google Scholar 

  45. J. Gwertzman and M. Seltzer. World-Wide Web cache consistency. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.

    Google Scholar 

  46. H. Hadimioglu and R. J. Flynn. The architectural design of a tightly-coupled distributed hypercube file system. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 147–150, 1989.

    Google Scholar 

  47. J. Hennessy and D. Patterson. Computer Architecture: A Quantitative Approach. Morgan Kaufmann, San Mateo, CA, 1990.

    Google Scholar 

  48. IEEE workshop on metadata for scientific and technical data management. See http://www.llnl.gov/livxomp/metadata/minutes/minutes-1994–05.html, May 1994.

    Google Scholar 

  49. R. Jain. Scheduling data transfers in parallel computers and communications systems. Technical Report TR-93–03, Univ. Texas at Austin, Dept. of Comp. Sci., Feb. 1993.

    Google Scholar 

  50. R. Jain and N. Krishnakumar. Asymmetric costs and dynamic query processing in mobile computing environments. In Proc. WINLAB Workshop, Rutgers Univ., Apr. 1995.

    Google Scholar 

  51. R. Jain, K. Somalwar, J. Werth, and J. C. Browne. Scheduling parallel I/O operations in multiple-bus systems. J. Par. and Distrib. Comp., Dec. 1992. Special Issue on Scheduling and Load Balancing.

    Google Scholar 

  52. R. Jain, K. Somalwar, J. Werth, and J. C. Browne. Heuristics for scheduling parallel I/O operations. IEEE Trans. Par. Distrib. Sys., 1996. To appear.

    Google Scholar 

  53. R. Jain and J. Werth. Airdisks and AirRAID: Modeling and scheduling periodic wireless data broadcast. DIMACS Tech. Report 95–11, Rutgers Univ., May 1995.

    Google Scholar 

  54. R. Jain and J. Werth. Airdisks and AirRAID: Modeling and scheduling periodic wireless data broadcast. ACM SIGARCH Comp. Arch. News., Oct. 1995.

    Google Scholar 

  55. R. Jain, J. Werth, J. C. Browne, I. Chakravarty, and P. Highnam. An experimental study of the effectiveness of high level parallel programming. In 5th SIAM Intl. Conf. Par. Proc, 1991.

    Google Scholar 

  56. R. Jain, J. Werth, J. C. Browne, and G. Sasaki. A graph-theoretic model for the scheduling problem and its application to simultaneous resource scheduling. In ORSA Conf. on Computer Science and Operations Research: New Developments in their Interfaces, Jan. 1992. Available from Pergamon Press.

    Google Scholar 

  57. W. Jilke. Disk array mass storage systems: The new opportunity. Technical report, Amperif Corp., Sep. 1986.

    Google Scholar 

  58. H.Jordan. Scalability of data transport. In Proc. Scalable High Perf. Computing Conf., pages 1–8, 1992.

    Chapter  Google Scholar 

  59. A. Kandappan. Data allocation and scheduling for parallel I/O systems. Master’s thesis, Dept. of Elect, and Comp. Eng., Univ. of Texas at Austin, 1990.

    Google Scholar 

  60. A. E. Kaufman, editor. Special Issue on Visualization. IEEE Computer, July 1994.

    Google Scholar 

  61. M. Y. Kim. Synchronized disk interleaving. IEEE Trans. Comp., C-35, 1986.

    Google Scholar 

  62. D. Kotz. Multiprocessor file system interfaces. In Proc. 2nd Intl. Conf. Par. Distrib. Info. Sys., pages 194–201, 1993.

    Google Scholar 

  63. D. Kotz. Disk-directed I/O for MIMD multiprocessors. Technical Report PCS-TR94–226, Dartmouth College, 1994.

    Google Scholar 

  64. David Kotz. Introduction to multiprocessor I/O architecture. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  65. O. Krieger and M. Stumm. The Alloc stream facility: A redesign of application-level stream I/O. IEEE Computer, pages 75–82, Mar. 1994.

    Google Scholar 

  66. N. Krishnakumar and R. Jain. Protocols for maintaining inventory databases and user service profiles in mobile sales applications. In Proc. MOBIDATA Workshop on Mobile and Wireless Info. Sys., Rutgers Univ., Nov. 1994.

    Google Scholar 

  67. J. M. Kumar and L. M. Patnaik. Extended hypercube: A hierarchical network of hypercubes. IEEE Trans. Par. Distrib. Comp., pages 45–57, Jan. 1992.

    Google Scholar 

  68. K. Li, R. Kumpf, P. Horton, and T. Anderson. A quantitative analysis of disk drive power management in portable computers. In Proc. Winter 1994 USENIX Conf., Jan. 1994.

    Google Scholar 

  69. M. Livny, S. Khoshhafian, and H. Boral. Multi-disk management algorithms. In Proc. SIGMETRICS, May 1987.

    Google Scholar 

  70. E. P. Markatos and G. Dramitinos. Implementation of a reliable remote memory pager. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.

    Google Scholar 

  71. E. Miller. Input/Output behavior of supercomputing applications. Technical Report UCB/CSD 91/616, Univ. California, Berekeley, 1991.

    Google Scholar 

  72. S. A. Moyer and V. Sunderam. Scalable concurrency control for parallel file systems. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  73. N. Nieuwejaar and D. Kotz. Low-level Interfaces for High-level Parallel I/O. In Inputl Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  74. N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-access characteristics of parallel scientific workloads. Tech. Rep. PCS-TR95–263, Dept. of Comp. Sci., Dartmouth Coll., 1995.

    Google Scholar 

  75. M. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. 26th Hawaii Intl. Conf Sys. Sci., pages 50–59, 1993.

    Google Scholar 

  76. J. Pasquale. I/O system design for intensive multimedia I/O. In Proc. IEEE Workshop on Workstation Op. Sys, Apr. 1992.

    Google Scholar 

  77. J. Pasquale. Systems software and hardware support considerations for digital video and audio computing. In Proc. 26th Hawaii Intl. Conf. Sys. Sci., page 15, 1993.

    Google Scholar 

  78. D. A. Patterson, G. A. Gibson, and R. H. Katz. A case for redundant arrays of inexpensive disks (RAID). In Proc. SIGMOD, 1988.

    Google Scholar 

  79. R. Hugo Patterson, G. A. Gibson, E. Ginting, D. Stodolsky, and J. Zelenka. Informed prefetching and caching. In Proc. ACM Symp. Op. Sys. Princ, Dec. 1995.

    Google Scholar 

  80. J. Pepper. Deploying a remote sales staff. Comm. Week, pages 23–24, Jan. 15, 1996.

    Google Scholar 

  81. P. Pierce. A concurrent file system for a highly parallel mass storage system. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 155–160, 1989.

    Google Scholar 

  82. T. Pratt, J. French, P. Dickens, and S. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 161–166, 1989.

    Google Scholar 

  83. A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proc. Intl. Par. Proc. Symp., pages 165–172, Apr. 1995.

    Google Scholar 

  84. A. L. N. Reddy and P. Banerjee. Design, analysis and simulation of I/O architectures for hypercube multiprocessors. IEEE Trans. Par. and Distrib. Sys., pages 140–151, Apr. 1990.

    Google Scholar 

  85. A. L. N. Reddy, P. Banerjee, and D. K. Chen. Compiler support for parallel I/O operations. Technical Report RJ 7918 (# 72901), IBM Almaden Research Center, 1991.

    Google Scholar 

  86. A. A. Rodriguez and L. A. Rowe, editors. Special Issue on Multimedia systems and applications. IEEE Computer, Mar. 1995.

    Google Scholar 

  87. M. W. Sachs, A. Leff, and D. Sevigny. LAN and I/O convergence: A survey of the issues. IEEE Computer, pages 24–32, Dec. 1994.

    Google Scholar 

  88. K. Salem and H. Garcia-Molina. Disk striping. In Proc. IEEE Intl. Conf. Data Eng., 1986.

    Google Scholar 

  89. J. Schwartz. Upgrade lets salespeople share data. Comm. Week, pages 47–48, May 23 1994.

    Google Scholar 

  90. O. Sharp. The grand challenges. Byte, pages 65–72, Feb. 1995.

    Google Scholar 

  91. Elizabeth Shriver and Mark Nodine. An introduction to parallel I/O models and algorithms. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

  92. J. E. Smith, W. C. Hsu, and C. Hsuing. Future general purpose supercomputer architectures. In Proc. Supercomp. ’90, pages 796–804, 1990.

    Book  Google Scholar 

  93. IEEE Spectrum. Special Issue on interactive multimedia. Mar. 1993.

    Google Scholar 

  94. P. Steenkiste. A systematic approach to host interface design for high speed networks. IEEE Computer, Mar. 1994.

    Google Scholar 

  95. R. Steinmetz. Synchronization properties in multimedia systems. IEEE J. Sel. Areas Comm., Apr. 1990.

    Google Scholar 

  96. M. Stonebraker and G. A. Schloss. Distributed RAID — a new multiple copy algorithm. In Proc. 6th Intl. Conf. Data Eng., pages 430–437, 1990.

    Chapter  Google Scholar 

  97. S. Viswanathan T. Imielinski and B.R. Badrinath. Energy efficient indexing on air. In Proc. SIGMOD, pages 25–36, 1994.

    Google Scholar 

  98. G. Tiwary. Reducing power consumption on ASICs. Computer Design, Mar. 1995.

    Google Scholar 

  99. C. B. S. Traw and J. M. Smith. Striping within the network subsytem. IEEE Network, pages 22–29, July/Aug. 1995.

    Google Scholar 

  100. J. S. Vitter and E. A. M. Shriver. Optimal disk I/O with parallel block transfer. In Proc. ACM Symp. Theory of Comp., 1990.

    Google Scholar 

  101. J. S. Vitter and E. A. M. Shriver. Algorithms for parallel memory I: Two-level memories. Algorithmica, pages 110–147, Aug./Sep. 1994.

    Google Scholar 

  102. J. S. Vitter and E. A. M. Shriver. Algorithms for parallel memory II: Hierarchical multilevel memories. Algorithmica, pages 148–169, Aug./Sep. 1994.

    Google Scholar 

  103. R. Wilson. ISSCC eyes a consumer, mobile future. Electronic Eng. Times, 1994.

    Google Scholar 

  104. D. Womble, D. Greenberg, S. Wheat, and R. Reisen. Beyond core: Making parallel computer I/O practical. In Proc. 1993 DAGS Symposium, June 1993.

    Google Scholar 

  105. Wm. A. Wulf and S. A. McKee. Hitting the memory wall: Implications of the obvious. ACM SIGARCH Comp. Arch. News, pages 20–24, Mar. 1995.

    Google Scholar 

  106. E. Wyatt. Batteries not included. Barron’s, Mar. 14, 1994.

    Google Scholar 

  107. H. Yokota and Y. Mimatsu. A scalable disk system with data reconstruction functions. In Input/Outputin Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Kluwer Academic Publishers

About this chapter

Cite this chapter

Jain, R., Werth, J., Browne, J.C. (1996). I/O in Parallel and Distributed Systems: An Introduction. In: Jain, R., Werth, J., Browne, J.C. (eds) Input/Output in Parallel and Distributed Computer Systems. The Kluwer International Series in Engineering and Computer Science, vol 362. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1401-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-1401-1_1

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4612-8607-3

  • Online ISBN: 978-1-4613-1401-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics