Zlatko Drmač
Skip Abstract Section
Skip Supplemental Material SectionAbstract
A new software for computing the singular value decomposition (SVD) of real or complex matrices is proposed. The method implemented in the code xGESVDQ is essentially the QR SVD algorithm available as xGESVD in LAPACK. The novelty is an extra step, the QR factorization with column (or complete row and column) pivoting, also already available in LAPACK as xGEQP3. For experts in matrix computations, the combination of the QR factorization and an SVD computation routine is not new. However, what seems to be new and important for applications is that the resulting procedure is numerically superior to xGESVD and that it is capable of reaching the accuracy of the Jacobi SVD. Further, when combined with pivoted Cholesky factorization, xGESVDQ provides numerically accurate and fast solvers (designated as xPHEVC, xPSEVC) for the Hermitian positive definite eigenvalue problem. For instance, using accurately computed Cholesky factor, xPSEVC computes all eigenvalues of the 200 × 200 Hilbert matrix (whose spectral condition number is greater that 10300) to nearly full machine precision. Furthermore, xGESVDQ can be used for accurate spectral decomposition of general (indefinite) Hermitian matrices.
Supplemental Material
Available for Download
zip
977.zip (11.3 MB)
Software for A QR--Preconditioned QR SVD Method for Computing the SVD with High Accuracy
- E. Anderson, Z. Bai, C. Bischof, L. S. Blackford, J. Demmel, J. J. Dongarra, J. Du Croz, S. Hammarling, A. Greenbaum, A. McKenney, and D. Sorensen. 1999. LAPACK Users’ Guide (3rd ed.). Society for Industrial and Applied Mathematics, Philadelphia, PA. Google Scholar
Cross Ref
- J. Barlow. 2002. More accurate bidiagonal reduction for computing the singular value decomposition. SIAM Journal on Matrix Analysis and Applications 23, 761--798. Google ScholarDigital Library
- J. Barlow and J. Demmel. 1990. Computing accurate eigensystems of scaled diagonally dominant matrices. SIAM Journal on Numerical Analysis 27, 3, 762--791. Google ScholarDigital Library
- J. L. Barlow, N. Bosner, and Z. Drmač. 2005. A new stable bidiagonal reduction algorithm. Linear Algebra and Its Applications 397, 35--84. Google ScholarCross Ref
- M. Bečka, G. Okša, and M. Vajteršic. 2015. New dynamic orderings for the parallel one-sided block-Jacobi SVD algorithm. Parallel Processing Letters 25, 1550003-1--1550003-19.Google ScholarCross Ref
- C. H. Bischof and G. Quintana-Orti. 1998a. Algorithm 782: Codes for rank-revealing QR factorizations of dense matrices. ACM Transactions on Mathematical Software 24, 2, 254--257. Google ScholarDigital Library
- C. H. Bischof and G. Quintana-Orti. 1998b. Computing rank-revealing QR factorizations of dense matrices. ACM Transactions on Mathematical Software 24, 2, 226--253. Google ScholarDigital Library
- N. Bosner and Z. Drmač. 2005. On accuracy properties of one-sided bidiagonalization algorithm and its applications. In Applied Mathematics and Scientific Computing, Z. Drmač, M. Marušić, and Z. Tutek, (Eds.). Springer, 141--150. Google ScholarCross Ref
- P. A. Businger and G. H. Golub. 1965. Linear least squares solutions by householder transformations. Numerische Mathematik 7, 269--276. Google ScholarDigital Library
- T. F. Chan. 1982. An improved algorithm for computing the singular value decomposition. ACM Transactions on Mathematical Software 8, 72--83. Google ScholarDigital Library
- A. J. Cox and N. J. Higham. 1998. Stability of Householder QR factorization for weighted least squares problems. In Numerical Analysis (Dundee). Pitman Research Notes in Mathematics, Vol. 380. Longman, Harlow, England, 57--73.Google Scholar
- J. Demmel. 1989. On Floating Point Errors in Cholesky. LAPACK Working Note 14. Computer Science Department, University of Tennessee.Google Scholar
- J. Demmel. 1997. Applied Numerical Linear Algebra. SIAM. Google ScholarCross Ref
- J. Demmel. 1999. Accurate singular value decompositions of structured matrices. SIAM Journal on Matrix Analysis and Applications 21, 2, 562--580. Google ScholarDigital Library
- J. Demmel, L. Grigori, M. Gu, and H. Xiang. 2015. Communication avoiding rank revealing QR factorization with column pivoting. SIAM Journal on Matrix Analysis and Applications 36, 1, 55--89. Google ScholarCross Ref
- J. Demmel, M. Gu, S. Eisenstat, I. Slapničar, K. Veselić, and Z. Drmač. 1999. Computing the singular value decomposition with high relative accuracy. Linear Algebra and Its Applications 299, 21--80. Google ScholarCross Ref
- J. Demmel and W. Kahan. 1990. Accurate singular values of bidiagonal matrices. SIAM Journal on Scientific and Statistical Computing 11, 5, 873--912. Google ScholarDigital Library
- J. Demmel and K. Veselić. 1992. Jacobi’s method is more accurate than QR. SIAM Journal on Matrix Analysis and Applications 13, 4, 1204--1245. Google ScholarDigital Library
- F. M. Dopico, J. M. Molera, and J. Moro. 2003. An orthogonal high relative accuracy algorithm for the symmetric eigenproblem. SIAM Journal on Matrix Analysis and Applications 25, 2, 301--351. Google ScholarDigital Library
- Z. Drmač. 1994. Computing the Singular and the Generalized Singular Values. Ph.D. Dissertation, Lehrgebiet Mathematische Physik, Fernuniversität Hagen, Germany.Google Scholar
- Z. Drmač. 1997. Implementation of Jacobi rotations for accurate singular value computation in floating point arithmetic. SIAM Journal on Scientific Computing 18, 1200--1222. Google ScholarDigital Library
- Z. Drmač. 1999. A posteriori computation of the singular vectors in a preconditioned Jacobi SVD algorithm. IMA Journal of Numerical Analysis 19, 191--213. Google ScholarCross Ref
- Z. Drmač. 2000. On principal angles between subspaces of Euclidean space. SIAM Journal on Matrix Analysis and Applications 22, 173--194. Google ScholarDigital Library
- Z. Drmač. 2015. SVD of Hankel matrices in Vandermonde-Cauchy product form. Electronic Transactions on Numerical Analysis 44, 593--623.Google Scholar
- Z. Drmač. 2016. xGESVDQ: A Software Implementation of the QR--Preconditioned QR SVD Method for Computing the Singular Value Decomposition User Guide. Technical Report. Department of Mathematics, Faculty of Science, University of Zagreb, Croatia.Google Scholar
- Z. Drmač and Z. Bujanović. 2008. On the failure of rank-revealing QR factorization software—a case study. ACM Transactions on Mathematical Software 35, 2, 12:1--12:28.Google ScholarDigital Library
- Z. Drmač and K. Veselić. 2000. Approximate eigenvectors as preconditioner. Linear Algebra and Its Applications 309, 13, 191--215. Google ScholarCross Ref
- Z. Drmač and K. Veselić. 2008a. New fast and accurate Jacobi SVD algorithm: I. SIAM Journal on Matrix Analysis and Applications 29, 4, 1322--1342. Google ScholarDigital Library
- Z. Drmač and K. Veselić. 2008b. New fast and accurate Jacobi SVD algorithm: II. SIAM Journal on Matrix Analysis and Applications 29, 4, 1343--1362. Google ScholarDigital Library
- J. A. Duersch and M. Gu. 2015. True BLAS-3 Performance QRCP Using Random Sampling. ArXiv e-prints.Google Scholar
- S. C. Eisenstat and I. C. F. Ipsen. 1995. Relative perturbation techniques for singular value problems. SIAM Journal on Numerical Analysis 32, 6, 1972--1988. Google ScholarDigital Library
- A. George, K. Ikramov, and A. B. Kucherov. 2000. Some properties of symmetric quasi-definite matrices. SIAM Journal on Matrix Analysis and Applications 21, 4, 1318--1323. Google ScholarDigital Library
- G. H. Golub and W. Kahan. 1965. Calculating the singular values and pseudo-inverse of a matrix. SIAM Journal on Numerical Analysis 2, 2, 205--224. Google ScholarCross Ref
- G. H. Golub and C. F. Van Loan. 1989. Matrix Computations (2nd ed.). Johns Hopkins University Press, Baltimore, MD.Google Scholar
- B. Großer and B. Lang. 2003. An O(n2) algorithm for the bidiagonal SVD. Linear Algebra and Its Applications 358, 13, 45--70. Google ScholarCross Ref
- N. J. Higham. 2000. QR factorization with complete pivoting and accurate computation of the SVD. Linear Algebra and Its Applications 309, 153--174. Google ScholarCross Ref
- Intel. 2015. Intel® Math Kernel Library 11.2 Update 3. Intel.Google Scholar
- W. Kahan. 2008. Why Can I Debug Some Numerical Programs That You Can’t? Retrieved June 6, 2017, from https://www.cs.berkeley.edu/∼wkahan/Stnfrd50.pdf.Google Scholar
- D. Knuth. 1977. The Correspondence Between Donald E. Knuth and Peter van Emde Boas on Priority Deques During the Spring of 1977. Retrieved June 6, 2017, from https://staff.fnwi.uva.nl/p.vanemdeboas/knuthnote.pdf.Google Scholar
- H. Ltaief, P. Luszczek, and J. Dongarra. 2013. High-performance bidiagonal reduction using tile algorithms on homogeneous multicore architectures. ACM Transactions on Mathematical Software 39, 3, 16:1--16:22.Google ScholarDigital Library
- M. J. D. Powell and J. K. Reid. 1969. On applying Householder transformations to linear least squares problems. In Information Processing 68: Proceedings of the International Federation of Information Processing Congress, Edinburgh, 1968. 122--126.Google Scholar
- R. Ralha. 2003. One-sided reduction to bidiagonal form. Linear Algebra and Its Applications 358, 13, 219--238. Google ScholarCross Ref
- G. W. Stewart. 1980. The efficient generation of random orthogonal matrices with an application to condition estimators. SIAM Journal of Numerical Analysis 17, 3, 403--409. Google ScholarCross Ref
- G. W. Stewart. 1995. QR Sometimes Beats Jacobi. Technical Report TR--3434. Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park.Google Scholar
- G. W. Stewart. 1997a. A Gap-Revealing Matrix Decomposition. Technical Report TR--3771. Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park.Google Scholar
- G. W. Stewart. 1997b. The QLP Approximation to the Singular Value Decomposition. Technical Report TR--97--75. Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park.Google Scholar
- G. W. Stewart and J. Sun. 1990. Matrix Perturbation Theory. Academic Press, Boston, MA.Google Scholar
- A. Tomas, Z. Bai, and V. Hernandez. 2012. Parallelization of the QR decomposition with column pivoting using column cyclic distribution on multicore and GPU processors. In High Performance Computing for Computational Science—VECPAR 2012. Lecture Notes in Computer Science, Vol. 7851. Springer, 50--58. Google ScholarCross Ref
- A. van der Sluis 1969. Condition numbers and equilibration of matrices. Numerische Mathematik 14, 14--23. Google ScholarDigital Library
- K. Veselić. 1996. A note on the accuracy of symmetric eigenreduction algorithms. Electronic Transactions on Numerical Analysis 4, 37--45.Google Scholar
- K. Veselić and V. Hari. 1989. A note on a one-sided Jacobi algorithm. Numerische Mathematik 56, 627--633. Google ScholarDigital Library
Index Terms
- Algorithm 977: A QR--Preconditioned QR SVD Method for Computing the SVD with High Accuracy
Recommendations
A Note on Jacobi Being More Accurate Than $QR$
In [SIAM J. Matrix Anal. Appl., 13 (1992) pp. 1204--1245], Demmel and Veselic present a theoretical and experimental analysis to show that the Jacobi method is more accurate than the $QR$ method when computing the eigenvalues of positive definite ...
Probabilistic Upper Bounds for the Matrix Two-Norm
We develop probabilistic upper bounds for the matrix two-norm, the largest singular value. These bounds, which are true upper bounds with a user-chosen high probability, are derived with a number of different polynomials that implicitly arise in the ...
The Singular Value Decomposition: Anatomy of Optimizing an Algorithm for Extreme Scale
The computation of the singular value decomposition, or SVD, has a long history with many improvements over the years, both in its implementations and algorithmically. Here, we survey the evolution of SVD algorithms for dense matrices, discussing the ...
Comments