Abstract
We sketch the reasons for the I/O bottleneck in parallel and distributed systems, pointing out that it can be viewed as a special case of a general bottleneck that arises at all levels of the memory hierarchy. We argue that because of its severity, the I/O bottleneck deserves systematic attention at all levels of system design. We then present a survey of the issues raised by the I/O bottleneck in five key areas of parallel and distributed systems: applications, algorithms, compilers, operating systems and architecture. Finally, we address some of the trends we observe emerging in new paradigms of parallel and distributed computing: the convergence of networking and I/O, I/O for massively distributed “global information systems” such as the World Wide Web, and I/O for mobile computing and wireless communications. These considerations suggest exciting new research directions in I/O for parallel and distributed systems in the years to come.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
A. Acharya, M. Uysal, R. Bennett, A. Mendelson, M. Beynon, J. Hollingsworth, J. Saltz, and A. Sussman. Tuning the performance of I/O-intensive applications. In Proc. Workshop on I/O in Par. and Distrib. Sys. (IOPADS), 1996.
S. Acharya, R. Alonso, M. Franklin, and S. Zdonik. Broadcast disks: Data management for asymmetric communication environments. In Proc. SIGMOD, 1995.
A. Aggarwal and J. S. Vitter. The Input/Output complexity of sorting and related problems. Comm. ACM, pages 1116–1127, Sep. 1988.
J. Akella and D. P. Siewiorek. Modeling and measurement of the impact of Input/Output on system performance. In Proc. 18th Intl. Symp. Comp. Arch., pages 390–399, 1991.
D. P. Anderson, Y. Osawa, and R. Govindan. Real-time disk storage and retrieval of digital audio and video. ACM Trans. Comp. Sys., 1993.
M. Arrott and S. Latta. Perspectives on visualization. IEEE Spectrum, pages 61–65, Sep. 1992.
B. R. Badrinath and T. Imielinski. Data management issues in mobile computing. In Wireless Datacomm ’92, 1992.
F. Baskett. Keynote address. Intl. Symp. Shared Mem. Proc., Apr. 1991.
Sandra Johnson Baylor, Caroline Benveniste, and Yarsun Hsu. Performance evaluation of a massively parallel I/O subsystem. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
Sandra Johnson Baylor and C. Eric Wu. Parallel I/O workload characteristics using Vesta. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
M. Bender et al. Unix for nomads: Making Unix support mobile computing. In Proc. USENIX Symp. Mobile and Location-Independent Comp. Symp., pages 53–67, Aug. 1993.
H.Boral and D.J.DeWitt. Database machines: An idea whose time has passed? A critique of the future of database machines. In Third Intl. Workshop on Database Machines, pages 166–187, 1983.
Rajesh Bordawekar and Alok Choudhary. Issues in compiling I/O intensive problems. In Input/Outputin Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
D. Bradley and D. A. Reed. Performance of the Intel iPSC/2 input/output system. In Proc. Conf. on Hypercubes, Concurrent Comp, and Appl., pages 141–144, 1990.
J. C. Browne, Muhammad Azam, and Stephen Sobek. CODE: A unified approach to parallel programming. IEEE Software, page 11, July 1989.
J. C. Browne, A. Dale, C. Leung, and R. Jenevein. A parallel multi-stage I/O architecture with self-managing disk cache for database management applications. In Fourth Intl. Workshop on Database Machines. Springer-Verlag, 1985.
J. C. Browne, G. E. Onstott, P. L. Soffa, Ron Goering, S. Sivaramakrishnan, Harish Balan, and K. Somalwar. Design and evaluation of external memory architectures for multiprocessor computer systems: Second quarter report to IBM Yorktown Heights Research Lab. Technical report, Univ. Texas at Austin, Dept. of Comp. Sci., 1987. Available from J. C. Browne.
C. E. Catlett. Balancing resources. IEEE Spectrum, pages 48–55, Sep. 1992.
A. Chankhunthod, P. Danzig, C. Neerdaels, M. Schwartz, and K. Worrell. A hierarchical Internet object cache. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.
J. Child. Making every watt count. Computer Design, page 67, Dec. 1993.
H.-K. Jerry Chu. Zero-copy TCP in Solaris. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.
S. A. Coleman and R. W. Watson. New architectures to reduce I/O bottlenecks in high-performance systems. In Proc. 26th Hawaii Intl. Conf. Sys. Sci., page 5, 1993.
P. Corbett, D. Feitelson, S. Fineberg, Y. Hsu, B. Nitzberg, J.-P. Prost, M. Snir, B. Traversat, and P. Wong. Overview of the MPI-IO parallel I/O interface. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
T. C. Cormen and K. Bruhl. Don’t be too clever: Routing BMMC permutations on the MasPar MP-2. In Proc. Symp. Par. Algm. Arch. (SPAA), July 1995.
T. C. Cormen and A. Colvin. ViC*: A preprocessor for virtual-memory C*. Tech. Rep. PCS-TR94–243, Dept. of Comp. Sci., Dartmouth, Nov. 1994.
T. C. Cormen and D. Kotz. Integrating theory and practice in parallel file systems. Tech. Rep. PCS-TR93–188, Dept. of Comp. Sci., Dartmouth, Nov. 1994.
P. Crandall, R. A. Aydt, A. A. Chien, and D. A. Reed. Input/output characteristics of scalable parallel applications. In Proc. Supercomp. ’95, Dec. 1995.
D. E. Culler, R. M. Karp, D. A. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. Log P: Towards a realistic model of parallel computation. In Proc. Symp. Princ. Prac. of Par. Prog., May 1993.
R. Cypher, A. Ho, S. Konstantinidou, and P. Messina. Architectural requirements of parallel scientific applications with explicit communication. In Intl. Symp. Comp. Arch., 1993.
Juan Miguel del Rosario, R. Bordawekar, and Alok Chaudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. Workshop on I/O in Parallel Computer Systems, pages 56–70, 1993. Also in ACM SIGARCH Comp. Arch. News., Dec. 1993.
Juan Miguel del Rosario and Alok Choudhary. High-performance I/O for massively parallel computers: Problems and prospects. IEEE Computer, pages 59–68, Mar. 1994.
D. De Witt and J. Gray. Parallel database systems: The future of high performance database systems. CACM, 35:85–98, June 1992.
P. Druschel, M. B. Abbott, M. Pagels, and L. L. Peterson. Network subsystem design. IEEE Network, pages 8–17, July 1993.
D. Durand, R. Jain, and D. Tseytlin. Improving the performance of parallel I/O using distributed scheduling algorithms. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
K. Fall and J. Pasquale. Exploiting in-kernel data paths to improve I/O throughput and CPU availability. In Proc. Winter USENIX, pages 327–333, Jan. 1993.
High Performance Fortran Forum. High performance Fortran language specification. Sci. Prog., 2:1–170, 1993.
E. A. Fox, editor. Special Issue on digital multimedia systems. Comm. ACM, Apr. 1991.
G. Fox, S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, and C. Tseng. Fortran D language specifications. Technical Report COMP TR90–141, Rice Univ., 1990.
J. C. French, T. W. Pratt, and M. Das. Performance meaurement of a parallel Input/Output system for the Intel iPSC/2 hypercube. In Proc. SIGMETRICS, pages 178–187, 1991.
D. J. Gemmell, H. M. Vin, D. D. Kandlur, P. Venkat Rangan, and L. A. Rowe. Multimedia storage servers: A tutorial. IEEE Computer, pages 40–49, Mar. 1995.
J. Ghosh and B. Agarwal. Parallel I/O subsystems for hypercube multicomputers. In Proc. Intl. Par. Proc. Symp., pages 381–384, 1991.
Joydeep Ghosh, K. D. Goveas, and J. T. Draper. Performance evaluation of a parallel I/O subsystem for hypercube multicomputers. J. Par. Distrib. Comp., Jan/Feb. 1993.
G. A. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. PhD thesis, Univ. of Calif., Berkeley, Comp. Sci. Div, 1990. Also available as Tech. Rep. UCB/CSD 91/613.
G. A. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. The MIT Press, 1992.
J. Gwertzman and M. Seltzer. World-Wide Web cache consistency. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.
H. Hadimioglu and R. J. Flynn. The architectural design of a tightly-coupled distributed hypercube file system. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 147–150, 1989.
J. Hennessy and D. Patterson. Computer Architecture: A Quantitative Approach. Morgan Kaufmann, San Mateo, CA, 1990.
IEEE workshop on metadata for scientific and technical data management. See http://www.llnl.gov/livxomp/metadata/minutes/minutes-1994–05.html, May 1994.
R. Jain. Scheduling data transfers in parallel computers and communications systems. Technical Report TR-93–03, Univ. Texas at Austin, Dept. of Comp. Sci., Feb. 1993.
R. Jain and N. Krishnakumar. Asymmetric costs and dynamic query processing in mobile computing environments. In Proc. WINLAB Workshop, Rutgers Univ., Apr. 1995.
R. Jain, K. Somalwar, J. Werth, and J. C. Browne. Scheduling parallel I/O operations in multiple-bus systems. J. Par. and Distrib. Comp., Dec. 1992. Special Issue on Scheduling and Load Balancing.
R. Jain, K. Somalwar, J. Werth, and J. C. Browne. Heuristics for scheduling parallel I/O operations. IEEE Trans. Par. Distrib. Sys., 1996. To appear.
R. Jain and J. Werth. Airdisks and AirRAID: Modeling and scheduling periodic wireless data broadcast. DIMACS Tech. Report 95–11, Rutgers Univ., May 1995.
R. Jain and J. Werth. Airdisks and AirRAID: Modeling and scheduling periodic wireless data broadcast. ACM SIGARCH Comp. Arch. News., Oct. 1995.
R. Jain, J. Werth, J. C. Browne, I. Chakravarty, and P. Highnam. An experimental study of the effectiveness of high level parallel programming. In 5th SIAM Intl. Conf. Par. Proc, 1991.
R. Jain, J. Werth, J. C. Browne, and G. Sasaki. A graph-theoretic model for the scheduling problem and its application to simultaneous resource scheduling. In ORSA Conf. on Computer Science and Operations Research: New Developments in their Interfaces, Jan. 1992. Available from Pergamon Press.
W. Jilke. Disk array mass storage systems: The new opportunity. Technical report, Amperif Corp., Sep. 1986.
H.Jordan. Scalability of data transport. In Proc. Scalable High Perf. Computing Conf., pages 1–8, 1992.
A. Kandappan. Data allocation and scheduling for parallel I/O systems. Master’s thesis, Dept. of Elect, and Comp. Eng., Univ. of Texas at Austin, 1990.
A. E. Kaufman, editor. Special Issue on Visualization. IEEE Computer, July 1994.
M. Y. Kim. Synchronized disk interleaving. IEEE Trans. Comp., C-35, 1986.
D. Kotz. Multiprocessor file system interfaces. In Proc. 2nd Intl. Conf. Par. Distrib. Info. Sys., pages 194–201, 1993.
D. Kotz. Disk-directed I/O for MIMD multiprocessors. Technical Report PCS-TR94–226, Dartmouth College, 1994.
David Kotz. Introduction to multiprocessor I/O architecture. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
O. Krieger and M. Stumm. The Alloc stream facility: A redesign of application-level stream I/O. IEEE Computer, pages 75–82, Mar. 1994.
N. Krishnakumar and R. Jain. Protocols for maintaining inventory databases and user service profiles in mobile sales applications. In Proc. MOBIDATA Workshop on Mobile and Wireless Info. Sys., Rutgers Univ., Nov. 1994.
J. M. Kumar and L. M. Patnaik. Extended hypercube: A hierarchical network of hypercubes. IEEE Trans. Par. Distrib. Comp., pages 45–57, Jan. 1992.
K. Li, R. Kumpf, P. Horton, and T. Anderson. A quantitative analysis of disk drive power management in portable computers. In Proc. Winter 1994 USENIX Conf., Jan. 1994.
M. Livny, S. Khoshhafian, and H. Boral. Multi-disk management algorithms. In Proc. SIGMETRICS, May 1987.
E. P. Markatos and G. Dramitinos. Implementation of a reliable remote memory pager. In Proc. USENIX 1996 Ann. Tech. Conf., Jan. 1996.
E. Miller. Input/Output behavior of supercomputing applications. Technical Report UCB/CSD 91/616, Univ. California, Berekeley, 1991.
S. A. Moyer and V. Sunderam. Scalable concurrency control for parallel file systems. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
N. Nieuwejaar and D. Kotz. Low-level Interfaces for High-level Parallel I/O. In Inputl Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-access characteristics of parallel scientific workloads. Tech. Rep. PCS-TR95–263, Dept. of Comp. Sci., Dartmouth Coll., 1995.
M. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. 26th Hawaii Intl. Conf Sys. Sci., pages 50–59, 1993.
J. Pasquale. I/O system design for intensive multimedia I/O. In Proc. IEEE Workshop on Workstation Op. Sys, Apr. 1992.
J. Pasquale. Systems software and hardware support considerations for digital video and audio computing. In Proc. 26th Hawaii Intl. Conf. Sys. Sci., page 15, 1993.
D. A. Patterson, G. A. Gibson, and R. H. Katz. A case for redundant arrays of inexpensive disks (RAID). In Proc. SIGMOD, 1988.
R. Hugo Patterson, G. A. Gibson, E. Ginting, D. Stodolsky, and J. Zelenka. Informed prefetching and caching. In Proc. ACM Symp. Op. Sys. Princ, Dec. 1995.
J. Pepper. Deploying a remote sales staff. Comm. Week, pages 23–24, Jan. 15, 1996.
P. Pierce. A concurrent file system for a highly parallel mass storage system. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 155–160, 1989.
T. Pratt, J. French, P. Dickens, and S. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Proc. Conf. on Hypercubes, Concurrent Comp. and Appl., pages 161–166, 1989.
A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proc. Intl. Par. Proc. Symp., pages 165–172, Apr. 1995.
A. L. N. Reddy and P. Banerjee. Design, analysis and simulation of I/O architectures for hypercube multiprocessors. IEEE Trans. Par. and Distrib. Sys., pages 140–151, Apr. 1990.
A. L. N. Reddy, P. Banerjee, and D. K. Chen. Compiler support for parallel I/O operations. Technical Report RJ 7918 (# 72901), IBM Almaden Research Center, 1991.
A. A. Rodriguez and L. A. Rowe, editors. Special Issue on Multimedia systems and applications. IEEE Computer, Mar. 1995.
M. W. Sachs, A. Leff, and D. Sevigny. LAN and I/O convergence: A survey of the issues. IEEE Computer, pages 24–32, Dec. 1994.
K. Salem and H. Garcia-Molina. Disk striping. In Proc. IEEE Intl. Conf. Data Eng., 1986.
J. Schwartz. Upgrade lets salespeople share data. Comm. Week, pages 47–48, May 23 1994.
O. Sharp. The grand challenges. Byte, pages 65–72, Feb. 1995.
Elizabeth Shriver and Mark Nodine. An introduction to parallel I/O models and algorithms. In Input/Output in Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
J. E. Smith, W. C. Hsu, and C. Hsuing. Future general purpose supercomputer architectures. In Proc. Supercomp. ’90, pages 796–804, 1990.
IEEE Spectrum. Special Issue on interactive multimedia. Mar. 1993.
P. Steenkiste. A systematic approach to host interface design for high speed networks. IEEE Computer, Mar. 1994.
R. Steinmetz. Synchronization properties in multimedia systems. IEEE J. Sel. Areas Comm., Apr. 1990.
M. Stonebraker and G. A. Schloss. Distributed RAID — a new multiple copy algorithm. In Proc. 6th Intl. Conf. Data Eng., pages 430–437, 1990.
S. Viswanathan T. Imielinski and B.R. Badrinath. Energy efficient indexing on air. In Proc. SIGMOD, pages 25–36, 1994.
G. Tiwary. Reducing power consumption on ASICs. Computer Design, Mar. 1995.
C. B. S. Traw and J. M. Smith. Striping within the network subsytem. IEEE Network, pages 22–29, July/Aug. 1995.
J. S. Vitter and E. A. M. Shriver. Optimal disk I/O with parallel block transfer. In Proc. ACM Symp. Theory of Comp., 1990.
J. S. Vitter and E. A. M. Shriver. Algorithms for parallel memory I: Two-level memories. Algorithmica, pages 110–147, Aug./Sep. 1994.
J. S. Vitter and E. A. M. Shriver. Algorithms for parallel memory II: Hierarchical multilevel memories. Algorithmica, pages 148–169, Aug./Sep. 1994.
R. Wilson. ISSCC eyes a consumer, mobile future. Electronic Eng. Times, 1994.
D. Womble, D. Greenberg, S. Wheat, and R. Reisen. Beyond core: Making parallel computer I/O practical. In Proc. 1993 DAGS Symposium, June 1993.
Wm. A. Wulf and S. A. McKee. Hitting the memory wall: Implications of the obvious. ACM SIGARCH Comp. Arch. News, pages 20–24, Mar. 1995.
E. Wyatt. Batteries not included. Barron’s, Mar. 14, 1994.
H. Yokota and Y. Mimatsu. A scalable disk system with data reconstruction functions. In Input/Outputin Parallel and Distributed Computer Systems. Kluwer Academic Publishers, 1996. (This volume).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Kluwer Academic Publishers
About this chapter
Cite this chapter
Jain, R., Werth, J., Browne, J.C. (1996). I/O in Parallel and Distributed Systems: An Introduction. In: Jain, R., Werth, J., Browne, J.C. (eds) Input/Output in Parallel and Distributed Computer Systems. The Kluwer International Series in Engineering and Computer Science, vol 362. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1401-1_1
Download citation
DOI: https://doi.org/10.1007/978-1-4613-1401-1_1
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4612-8607-3
Online ISBN: 978-1-4613-1401-1
eBook Packages: Springer Book Archive