Elsevier

Handbook of Statistics

Volume 9, 1993, Pages 303-376
Handbook of Statistics

9 Numerical aspects of solving linear least squares problems

https://doi.org/10.1016/S0169-7161(05)80133-6Get rights and content

Publisher Summary

This chapter explains some matrix computations that are common in statistics. Most of these techniques also arise in other applications, including the solution of partial differential equations and networking problems. Three types of errors are input or representation errors, approximation errors, and rounding errors. The chapter discusses the two methods used for analyzing rounding errors (1) forward error analysis and (2) backward error analysis. Backward error analysis separates the analysis of the solution of g into two separate problems. The first is to show that a particular algorithm satisfies and the second is to show that a bound of the form exists. It is important that an algorithm is efficient and accurate. There is always a tradeoff between efficiency and accuracy. The chapter also discusses several methods for solving least squares problems and some basics of floating point arithmetic. Very rarely does a program need to deal with numbers that are too large for the number range of a reasonable computer. Thus, an overflow probably means an error in the data, program, or algorithm.

References (120)

  • ChanT.F.

    Rank revealing QR factorization

    Linear Algebra Appl.

    (1987)
  • GentlemanW.M. et al.

    Matrix triangularization by systolic arrays

  • AndersonE. et al.

    LAPACK User's Guide

    (1992)
  • ArioliM. et al.

    On the augmented system approach to sparse linear least-squares problems

  • BareissE.H.

    Numerical solution of weighted least squares problems by G-transformations

  • BarlowJ.L.

    Probabilistic error analysis of floating point and CRD arithmetics

  • BarlowJ.L.

    Stability analysis of the G-algorithm and a note on its application to sparse least squares problems

    BIT

    (1985)
  • BarlowJ.L.

    Error analysis and implementation aspects of deferred correction for equality constrained least squares problems

    SIAM J. Numer. Anal.

    (1988)
  • BarlowJ.L.

    Error analysis of update methods for the symmetric eigenvalue problem

    SIAM J. Matrix Anal. Appl.

    (1993)
  • BarlowJ.L.

    Error analysis of a pairwise summation algorithm to compute the sample variance

    Numer. Math.

    (1991)
  • BarlowJ.L. et al.

    Computing accurate eigensystems of scaled diagonally dominant matrices

    SIAM J. Numer. Anal.

    (1990)
  • BarlowJ.L. et al.

    The direct solution of weighted and equality constrained least squares problems

    SIAM J. Sci. Statist. Comput.

    (1988)
  • BarlowJ.L. et al.

    Scaled Givens rotations for the solution of linear least squares problems on systolic arrays

    SIAM J. Sci. Statist. Comput.

    (1987)
  • BarlowJ.L. et al.

    Iterative methods for equality constrained least squares problems

    SIAM J. Sci. Statist. Comput.

    (1988)
  • BarlowJ.L. et al.

    A note on deferred correction for equality constrained least squares problems

    SIAM J. Numer. Anal.

    (1992)
  • BarlowJ.L. et al.

    Rank detection methods for sparse matrices

    SIAM J. Matrix Anal. Appl.

    (1992)
  • BischofC.H.

    Incremental condition estimation

    SIAM J. Matrix Anal. Appl.

    (1990)
  • BischofC.H.

    Private communication

  • BischofC.H. et al.

    Incremental condition estimation for sparse matrices

    SIAM J. Matrix Anal. Appl.

    (1990)
  • BjörckÅ.

    Solving linear least squares problems by Gram-Schmidt orthogonalization

    BIT

    (1967)
  • BjörckÅ.

    A general updating algorithm for constrained linear least squares problems

    SIAM J. Sci. Statist. Comput.

    (1984)
  • BjörckÅ.

    Least Squares Methods

    (1989)
  • BjörckÅ.

    Pivoting and stability in the augmented system method

  • BjörckÅ. et al.

    Iterative refinement of linear least squares solutions by Householder transformations

    BIT

    (1967)
  • BlueJ.L.

    A portable FORTRAN program to find the Euclidean norm of a vector

    ACM Trans. Math. Software

    (1978)
  • BroydenC.G.

    Some condition number bounds for the Gaussian elimination process

    J. Inst. Math. Appl.

    (1993)
  • BunchJ.R. et al.

    Rank-one modification of the symmetric eigenproblem

    Numer. Math.

    (1978)
  • BunchJ.R. et al.

    Direct methods for solving symmetric indefinite systems of linear equations

    SIAM J. Numer. Anal.

    (1971)
  • BusingerP.A. et al.

    Linear least squares solutions by house-holder transformations

    Numer. Math.

    (1965)
  • ChanT.F. et al.

    Algorithms for computing the sample variance: Analysis and recommendations

    Amer. Statist.

    (1983)
  • ChanT.F. et al.

    Computing truncated singular value decomposition least squares solution by rank revealing QR-factorizations

    SIAM J. Sci. Statist. Comput.

    (1990)
  • ChanT.F. et al.

    Computing standard deviations: Accuracy

    Comm. ACM

    (1979)
  • ClineA.K. et al.

    An estimate for the condition number of a matrix

    SIAM J. Numer. Anal.

    (1979)
  • CodyW.J.

    Static and dynamic numerical characteristics of floating point arithmetic

    IEEE Trans. Comput.

    (1973)
  • ConcusP. et al.

    A generalized conjugate gradient procedure for the numerical solution of elliptic partial differential equations

  • CuppenJ.

    A divide and conquer method for the symmetric tridiagonal eigen problem

    Numer. Math.

    (1981)
  • DemmelJ.W.

    Underflow and the reliability of numerical software

    SIAM J. Sci. Statist. Comput.

    (1984)
  • DemmelJ.W. et al.

    A prospectus for the development of a linear algebra library for high-performance computers

  • DemmelJ.W. et al.

    Accurate singular values of bidiagonal matrices

    SIAM J. Sci. Statist. Comput.

    (1990)
  • DemmelJ.W. et al.

    Jacobi's method is more accurate than QR

    SIAM J. Matrix Anal. Appl.

    (1992)
  • DongarraJ.J. et al.

    LINPACK User's Guide

    (1979)
  • DuffI.S. et al.

    Direct Methods for Sparse Matrices

    (1986)
  • EldénL.

    Perturbation theory for the least squares problem with equality constraints

    SIAM J. Numer. Anal.

    (1980)
  • EldénL. et al.

    An application of systolic arrays to discrete ill-posed problems

    SIAM J. Sci. Statist. Comput.

    (1986)
  • FosterL.V.

    Rank and null space calculations using matrix decompositions without column pivoting

    Linear Algebra. Appl.

    (1986)
  • FosterL.V.

    The probability of large diagonal elements in the QR factorization

    SIAM J. Sci. Statist. Comput.

    (1990)
  • FrancisJ.G.F.

    The QR transformation: A unitary analogue to the LR transformation, Parts I and II

    Comput. J.

    (1961)
    FrancisJ.G.F.

    The QR transformation: A unitary analogue to the LR transformation, Parts I and II

    Comput. J.

    (1961)
  • FrostO.L.

    An algorithm for linear constrained adaptive array processing

  • GaussC.F.

    Theoria Motus Corporum Coelestium in Sectionibus Conicus Solem Ambientium

    (1809)
  • GentlemanW.M.

    Least squares computations by Givens rotations without square roots

    Linear Algebra Appl.

    (1973)

    • Cited by (0)

    View full text
    Copyright © 1993 Published by Elsevier B.V.