Summary
This chapter discusses the design of modern numerical linear algebra problem solving environments. Particular emphasis is placed on three essential components out of which such environments are constructed, namely well-designed numerical software libraries, software tools that generate optimized versions of a collection of numerical kernels for various processor architectures, and software systems that transform disparate, loosely-connected computers and software libraries into a unified, easy-to-access computational service.
A brief description of the “pioneers”, namely the EISPACK and LINPACK software libraries as well as their successor, the Linear Algebra PACKage (LAPACK), illustrates the essential importance of block-partitioned algorithms for shared-memory, vector, and parallel processors. Indeed, these algorithms reduce the frequency of data movement between different levels of hierarchical memory. A key idea in this approach is the use of the Basic Linear Algebra Subprograms (BLAS) as computational building blocks. An outline of the ScaLAPACK software library, which is a distributed-memory version of LAPACK, highlights the equal importance of the above design principles to the development of scalable algorithms for MIMD distributed-memory concurrent computers. The impact of the architecture of high performance computers on the design of such libraries is stressed.
Producing hand-optimized implementations of even a reduced set of well designed software components such as the BLAS for a wide range of architectures is an expensive and tedious proposition. For any given architecture, customizing a numerical kernel’s source code to optimize performance requires a comprehensive understanding of the exploitable hardware resources of that architecture. Since this time-consuming customization process must be repeated whenever a slightly different target architecture is available, the relentless pace of hardware innovation makes the tuning of numerical libraries a constant burden. This chapter presents an innovative approach to automating the process of producing such optimized kernels for various processor architectures.
Finally, many scientists and researchers increasingly tend nowadays to use simultaneously a variety of distributed computing resources such as massively parallel processors, networks and clusters of workstations and “piles” of PCs. This chapter describes the NetSolve software system that has been specifically designed and conceived to efficiently use such a diverse and lively computational environment and to tackle the problems posed by such a complex and innovative approach to scientific problem solving. NetSolve provides the user with a pool of computational resources. These resources are computational servers that provide run-time access to arbitrary optimized numerical software libraries. This unified, easy-to-access computational service can make enormous amounts of computing power transparently available to users on ordinary platforms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson E., Bai Z., Bischof C., Demmel J., Dongarra J., Du Croz J., Greenbaum A., Hammarling S., McKenney A., Ostrouchov S. and Sorensen D., LAPACK User’s Guide ( second edition ), SIAM, Philadelphia PA, 1995.
Anderson E. and Dongarra J., Results from the Initial Release of LAPACK, LAPACK Working Note, No. 16, Technical Report, University of Tennessee, Knoxville, TN, 1989.
Anderson E. and Dongarra J., Evaluating Block Algorithm Variants in LAPACK, LAPACK Working Note, No. 19, Technical Report, University of Tennessee, Knoxville, TN, 1990.
Bilmes J., Asanovic K., Chin C. W. and Demmel J., Optimizing Matrix Multiply Using PHiPAC: A Portable, High-Performance, ANSI C Coding Methodology, Proceedings of the International Conference on Supercomputing, ACM SIGARC, Vienna, Austria, 1997, 340–347.
Blackford L., Choi J., Cleary A., D’Azevedo E., Demmel J., Dhillon I., Dongarra J., Hammarling S., Henry G., Petitet A., Stanley K., Walker D. and Whaley R. C., ScaLAPACK Users’ Guide, SIAM, Philadelphia PA, 1997.
Blackford L., Dongarra J., Papadopoulos C., and Whaley R. C., Installation Guide and Design of the HPF 1.1 Interface to ScaLAPACK, SLHPF, LAPACK Working Note, No. 137, Technical Report UT CS-98–396, University of Tennessee, Knoxville, TN, 1998.
Blackford L. S. and Whaley R. C., ScaLAPACK Evaluation and Performance at the DoD MSRCs, LAPACK Working Note, No. 136, Technical Report UT CS-98–388, University of Tennessee, Knoxville, TN, 1998.
Browne S., Casanova H. and Dongarra J., Providing Access to High Performance Computing Technologies, in: Wasniewski J., Dongarra J., Madsen K. and Olesen D. (eds.), Lecture Notes in Computer Science No. 1184, Springer-Verlag, Berlin, 1996, 123–133.
Casanova H. and Dongarra J., NetSolve: A Network Server for Solving Computational Science Problems, Technical report UT CS-95–313, University of Tennessee, Knoxville, TN, 1995.
Choi J., Dongarra J., Ostrouchov S., Petitet A., Walker D. and Whaley R. C., A Proposal for a Set of Parallel Basic Linear Algebra Subprograms, LAPACK Working Note, No. 100, Technical report UT CS-95292, University of Tennessee, Knoxville, TN, 1995.
Culler D., Dusseau A., Goldstein S., Krishnamurthy A., Lumetta S., von Eicken T. and Yelick K., Introduction to Split-C: Version 0.9, Computer Science Division — EECS, University of California, Berkeley, CA 94720, 1993.
Dayde M., Duff I. and Petitet A., A Parallel Block Implementation of Level 3 BLAS for MIMD Vector Processors, ACM Transactions on Mathematical Software 20, 1994, 178–193.
Demmel J., LAPACK: A Portable Linear Algebra Library for Supercomputers, Proceedings of the 1989 IEEE Control Systems Society Workshop on Computer-Aided Control System Design, 1989.
Dongarra J., Increasing the Performance of Mathematical Software through High-Level Modularity, Proceedings Sixth Int. Symp. Comp. Methods in Eng. e4 Applied Sciences, Versailles, France, North-Holland, 1984, 239–248.
Dongarra J., Du Croz J., Hammarling S. and Duff I., A Set of Level 3 Basic Linear Algebra Subprograms, ACM Transactions on Mathematical Software 16, 1990, 1–17.
Dongarra J., Du Croz J., Hammarling S. and Hanson R., An Extended Set of Fortran Basic Linear Algebra Subroutines, A CM Transactions on Mathematical Software 14, 1988, 18–32.
Dongarra J., Duff I., Sorensen D. and Van der Vorst H., Solving Linear Systems on Vector and Shared Memory Computers, SIAM Publications, Philadelphia, PA, 1991.
Dongarra J. and Grosse E., Distribution of Mathematical Software via Electronic Mail, Communications of the ACM 30, 1987, 403–407.
Dongarra J., Mayes P. and Radicatidi Brozolo G., The IBM RISC System 6000 and Linear Algebra Operations, Supercomputer 8, 1991, 15–30.
Dongarra J., Pozo R. and Walker D., An Object Oriented Design for High Performance Linear Algebra on Distributed Memory Architectures, Proceedings of the Object Oriented Numerics Conference, 1993, 257–264.
Du Croz J. and Pont M., The Development of a Floating-Point Validation Package, Proceedings of the 8th Symposium on Computer Arithmetic, IEEE Computer Society Press, Como, Italy, 1987.
Eckel G., Neider J. and Hassler E., ImageVision Library Programming Guide, Silicon Graphics, Inc., Mountain View, CA, 1996.
Edelman A., Large Dense Numerical Linear Algebra in 1993: The Parallel Computing Influence, International Journal of Supercomputing Applications 7, 1993, 113–128.
Felten E. and Otto S., Coherent Parallel C, Proceedings of the Third Conference on Hypercube Concurrent Computers and Applications, ACM Press, 1988.
Foster I. and Kesselman C. (eds.), The Grid - Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers, Inc., San Francisco, CA, 1998.
Fox G., Johnson M., Lyzenga G., Otto S., J. Salmon and D. Walker, Solving Problems on Concurrent Processors, Volume 1, Prentice Hall, Englewood Cliffs, N.J., 1988.
Gallivan K., Plemmons R. and Sameh A., Parallel Algorithms for Dense Linear Algebra Computations, SIAM Review 32, 1990, 54–135.
Geisti A., Beguelin A., Dongarra J., Jiang W., Manchek R. and Sunderam V., PVM: Parallel Virtual Machine. A Users’ Guide and Tutorial for Networked Parallel Computing, The MIT Press Cambridge, Massachusetts, 1994.
Gupta A. and Kumar V., On the Scalability of FFT on Parallel Computers, Proceedings of the Frontiers 90 Conference on Massively Parallel Computation, IEEE Computer Society Press, 1990.
Harrington R., Origin and Development of the Method of Moments for Field Computation, IEEE Antennas and Propagation Magazine, 1990.
Hess J., Panel Methods in Computational Fluid Dynamics, Annal Reviews of Fluid Mechanics 22, 1990, 255–274.
Hess J. and Smith M., Calculation of Potential Flows about Arbitrary Bodies, in: Küchemann D., editor, Progress in Aeronautical Sciences, Volume 8, Pergamon Press, 1967.
Hockney R. W. and Jesshope C. R., Parallel Computers, Adam Hilger Ltd., Bristol, UK, 1981.
Kâgström B., Ling P. and Van Loan C., Portable High Performance GEMM-based Level 3 BLAS, in: Sincovec R. F. et al., (eds.), Parallel Processing for Scientific Computing, SIAM, Philadelphia, 1993, 339–346.
Kahan W., Paranoia, (See http://www.netlib.org/).
Koebel C., Loveman D., Schreiber R., Steele G., and Zosel M., The High Performance Fortran Handbook, The MIT Press, Cambridge, Massachusetts, 1994.
Lawson C., Hanson R., Kincaid D. and Krogh F., Basic Linear Algebra Subprograms for Fortran Usage, ACM Transactions Mathematical Software 5, 1979, 308–323.
Litzkow M. and Livny M. and Mutka M. W., Condor–A Hunter of Idle Workstations, Proceedings of the 8th International Conference of Distributed Computing Systems, 1988, 104–111.
The Math Works Inc., MATLAB Reference Guide,The Math Works Inc., 1992.
Message Passing Interface Forum, MPI: A Message-Passing Interface standard, International Journal of Supercomputer Applications 8, 1994, 159–416.
Oberhuber M., Integrating ImageVision into NetSolve, see http://www.icg.tu-graz.ac.at/mober/pub, 1997.
Sekiguchi S., Sato M., Nakada H., Matsuoka S. and Nagashima U., Ninf: Network based Information Library for Globally High Performance Computing, Proceedings of Parallel Object-Oriented Methods and Applications (POOMA), Santa Fe, 1996.
Snir M., Otto S., Huss-Lederman S., Walker D. and Dongarra J., MPI: The Complete Reference, MIT Press, Cambridge, Massachusetts, 1996.
Toledo S., Improving Instruction–Level Parallelism in Sparse Matrix–Vector Multiplication Using Reordering, Blocking, and Prefetching, Proceedings of the 8th SIAM Conference on Parallel Processing for Scientific Computing,SIAM, 1997, ISBN 0–89871–395–1 (CD–ROM).
Wang J., Generalized Moment Methods in Electromagnetics, John Wiley & Sons, New-York, 1991.
Whaley R. C. and Dongarra J., A User’s Guide to the BLACS v1.1, LAPACK Working Note, No. 94, Technical Report UT CS-95–281, University of Tennessee, Knoxville, 1995, ( See also http://www.netlib.org/blacs/).
Whaley R. C. and Dongarra J., Automatically Tuned Linear Algebra Software, Proceedings of Supercomputing ‘88, ACM SIGARCH and IEEE Computer Society, ISBN 0–89791–984-X (CD-ROM) (also LAPACK Working Note, No. 131, Technical Report UT CS-97–366, University of Tennessee, Knoxville, TN, 1997, ( See also http://www.net1ib.org/atlas/).
Wilkinson J., Reinsch C., Handbook for Automatic Computation: Volume II - Linear Algebra, Springer-Verlag, New York, 1971.
Wolfram S., The Mathematica Book, ( Third Edition ), Wolfram Median, Inc. and Cambridge University Press, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Petitet, A., Casanova, H., Dongarra, J., Robert, Y., Whaley, R.C. (2000). Parallel and Distributed Scientific Computing. In: Błażewicz, J., Ecker, K., Plateau, B., Trystram, D. (eds) Handbook on Parallel and Distributed Processing. International Handbooks on Information Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-04303-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-662-04303-5_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-08571-0
Online ISBN: 978-3-662-04303-5
eBook Packages: Springer Book Archive