Abstract
Underdetermined systems of equations in which the minimum norm solution needs to be computed arise in many applications, such as geophysics, signal processing, and biomedical engineering. In this article, we introduce a new parallel algorithm for obtaining the minimum 2-norm solution of an underdetermined system of equations. The proposed algorithm is based on the Balance scheme, which was originally developed for the parallel solution of banded linear systems. The proposed scheme assumes a generalized banded form where the coefficient matrix has column overlapped block structure in which the blocks could be dense or sparse. In this article, we implement the more general sparse case. The blocks can be handled independently by any existing sequential or parallel QR factorization library. A smaller reduced system is formed and solved before obtaining the minimum norm solution of the original system in parallel. We experimentally compare and confirm the error bound of the proposed method against the QR factorization based techniques by using true single-precision arithmetic. We implement the proposed algorithm by using the message passing paradigm. We demonstrate numerical effectiveness as well as parallel scalability of the proposed algorithm on both shared and distributed memory architectures for solving various types of problems.
- Patrick R. Amestoy, Iain S. Duff, and Chiara Puglisi. 1996. Multifrontal QR factorization in a multiprocessor environment. Numerical Linear Algebra with Applications 3, 4 (1996), 275--300. Google ScholarCross Ref
- Cleve Ashcraft and Roger G. Grimes. 1999. SPOOLES: An object-oriented sparse matrix library. In Proceedings of the 9th SIAM Conference on Parallel Processing for Scientific Computing. SIAM.Google Scholar
- Åke Björck. 1996. Numerical Methods for Least Squares Problems. SIAM, Philadelphia. Google ScholarCross Ref
- L. Susan Blackford, Jaeyoung Choi, Andy Cleary, Eduardo D’Azevedo, James Demmel, Inderjit Dhillon, Jack Dongarra, Sven Hammarling, Greg Henry, Antoine Petitet, and others. 1997. ScaLAPACK Users’ Guide. Vol. 4. SIAM. Google ScholarCross Ref
- Ronald F. Boisvert, Roldan Pozo, Karin A. Remington, Richard F. Barrett, and Jack Dongarra. 1996. Matrix market: A web resource for test matrix collections. In Quality of Numerical Software. 125--137.Google Scholar
- Randall Bramley and B. Winnicka. 1996. Solving linear inequalities in a least squares sense. SIAM Journal on Scientific Computing 17, 1 (1996), 275--286. DOI:http://dx.doi.org/10.1137/0917020 Google ScholarDigital Library
- Alfred Bruckstein, David Donoho, and Michael Elad. 2009. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review 51, 1 (2009), 34--81. DOI:http://dx.doi.org/10.1137/060657704 Google ScholarDigital Library
- Alfredo Buttari. 2013. Fine-grained multithreading for the multifrontal QR factorization of sparse matrices. SIAM Journal on Scientific Computing 35, 4 (2013), C323--C345. Google ScholarCross Ref
- Shane F. Cotter, Bhaskar D. Rao, Kjersti Engan, and Kenneth Kreutz-Delgado. 2005. Sparse solutions to linear inverse problems with multiple measurement vectors. Signal Processing, IEEE Transactions on 53, 7 (2005), 2477--2488. Google ScholarDigital Library
- Timothy A. Davis. 2009. Users guide for SuiteSparseQR, a multifrontal multithreaded sparse QR factorization package. http://faculty.cse.tamu.edu/davis/suitesparse.html.Google Scholar
- Timothy A. Davis. 2011. Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization. ACM Transactions on Mathematical Software (TOMS) 38, 1 (2011), 8.Google ScholarDigital Library
- Timothy A. Davis. 2013. SuiteSparse packages, released 4.2.1. http://faculty.cse.tamu.edu/davis/suitesparse.html.Google Scholar
- Timothy A. Davis and Yifan Hu. 2011. The university of florida sparse matrix collection. ACM Transactions on Mathematical Software (TOMS) 38, 1 (2011), 1.Google ScholarDigital Library
- Achiya Dax. 2008. A hybrid algorithm for solving linear inequalities in a least squares sense. Numerical Algorithms 50, 2 (2008), 97--114. DOI:http://dx.doi.org/10.1007/s11075-008-9218-3 Google ScholarCross Ref
- James W. Demmel and Nicholas J. Higham. 1993. Improved error bounds for underdetermined system solvers. SIAM Journal on Matrix Analysis and Applications 14, 1 (1993), 1--14. Google ScholarDigital Library
- Robert Fourer. 1982. Solving staircase linear programs by the simplex method, 1: Inversion. Mathematical Programming 23, 1 (1982), 274--313. Google ScholarCross Ref
- Robert Fourer. 1983. Solving staircase linear programs by the simplex method, 2: Pricing. Mathematical Programming 25, 3 (1983), 251--292. Google ScholarCross Ref
- Robert Fourer. 1984. Staircase matrices and systems. SIAM Review 26, 1 (1984), 1--70. Google ScholarDigital Library
- Leibniz Supercomputing Centre GCS Supercomputer. 2012. SuperMUC Petascale System. Retrieved from www.lrz.de.Google Scholar
- Philip E. Gill and Walter Murray. 1973. A numerically stable form of the simplex algorithm. Linear Algebra and Its Applications 7, 2 (1973), 99--138. Google ScholarCross Ref
- C. Roger Glassey and Peter Benenson. 1975. Quadratic Programming Analysis of Energy in the United States Economy. Final Report. Technical Report. University of California, Berkeley.Google Scholar
- Gene Golub, Ahmed H. Sameh, and Vivek Sarin. 2001. A parallel balance scheme for banded linear systems. Numerical Linear Algebra with Applications 8, 5 (2001), 285--299. Google ScholarCross Ref
- William Gropp, Ewing Lusk, Nathan Doss, and Anthony Skjellum. 1996. A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing 22, 6 (1996), 789--828. Google ScholarDigital Library
- Shih-Ping Han. 1980. Least-Squares Solution of Linear Inequalities. Technical Report. DTIC Document.Google Scholar
- Nicholas J. Higham. 1991. Algorithm 694: A collection of test matrices in MATLAB. ACM Transactions on Mathematical Software (TOMS) 17, 3 (1991), 289--305. Google ScholarDigital Library
- Nicholas J. Higham. 2002. Accuracy and Stability of Numerical Algorithms. SIAM. Google ScholarCross Ref
- James K. Ho. 1975. Optimal design of multi-stage structures: A nested decomposition approach. Computers & Structures 5, 4 (1975), 249--255. Google ScholarCross Ref
- James K. Ho and Etienne Loute. 1981. A set of staircase linear programming test problems. Mathematical Programming 20, 1 (1981), 245--250. DOI:http://dx.doi.org/10.1007/BF01589349 Google ScholarCross Ref
- He Huang, John M. Dennis, Liqiang Wang, and Po Chen. 2013. A scalable parallel LSQR algorithm for solving large-scale linear system for tomographic problems: A case study in seismic tomography. Procedia Computer Science 18 (2013), 581--590. Google ScholarCross Ref
- George Karypis and Vipin Kumar. 2009. Metis: Unstructured Graph Partitioning and Sparse Matrix Ordering System, Version 4.0. http://www.cs.umn.edu/∼metis.Google Scholar
- Charles L. Lawson and Richard J. Hanson. 1987. Solving Least Squares Problems (Classics in Applied Mathematics). Society for Industrial Mathematics.Google Scholar
- MATLAB. 2015. version 8.6.0 (R2015b). The MathWorks Inc., Natick, Massachusetts.Google Scholar
- Kenji Matsuura and Y. Okabe. 1995. Selective minimum-norm solution of the biomagnetic inverse problem. IEEE Transactions on Biomedical Engineering 42, 6 (June 1995), 608--615. DOI:http://dx.doi.org/10.1109/10.387200 Google ScholarCross Ref
- Mohammad Javad Peyrovian and Alexander A. Sawchuk. 1978. Image restoration by spline functions. Applied Optics 17, 4 (Feb 1978), 660--666. DOI:http://dx.doi.org/10.1364/AO.17.000660 Google ScholarCross Ref
- Mustafa Ç. Pinar. 1998. Newton’s method for linear inequality systems. European Journal of Operational Research 107, 3 (1998), 710--719. Google ScholarCross Ref
- James Reinders. 2007. Intel Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism. O’Reilly Media, Inc.Google Scholar
- Ahmed H. Sameh and Vivek Sarin. 2002. Parallel algorithms for indefinite linear systems. Parallel Computing 28, 2 (2002), 285--299. Google ScholarDigital Library
- Michael A. Saunders. 1972. Large-scale linear programming using the cholesky factorization. (1972).Google Scholar
- Mrinal K. Sen and Paul L. Stoffa. 2013. Global Optimization Methods in Geophysical Inversion. Cambridge University Press. Google ScholarCross Ref
- Tayfun E. Tezduyar and Ahmed Sameh. 2006. Parallel finite element computations in fluid mechanics. Computer Methods in Applied Mechanics and Engineering 195, 13--16 (2006), 1872--1884.Google Scholar
- Jia-Zhu Wang, Samuel J. Williamson, and Lloyd Kaufman. 1992. Magnetic source images determined by a lead-field analysis: The unique minimum-norm least-squares estimation. IEEE Transactions on Biomedical Engineering 39, 7 (1992), 665--675. Google ScholarCross Ref
- Michael S. Zhdanov. 2002. Geophysical Inverse Theory and Regularization Problems. Vol. 36. Elsevier. Google ScholarCross Ref
Index Terms
- Parallel Minimum Norm Solution of Sparse Block Diagonal Column Overlapped Underdetermined Systems
Recommendations
Structure-adaptive parallel solution of sparse triangular linear systems
Highlights- We develop a novel parallel algorithm for solution of sparse triangular linear systems.
AbstractSolving sparse triangular systems of linear equations is a performance bottleneck in many methods for solving more general sparse systems. Both for direct methods and for many iterative preconditioners, it is used to solve the system ...
Efficient parallel solution of sparse eigenvalue and eigenvector problems
FOCS '95: Proceedings of the 36th Annual Symposium on Foundations of Computer ScienceThis paper gives a new algorithm for computing the characteristic polynomial of a symmetric sparse matrix. We derive an interesting algebraic version of nested dissection, which constructs a sparse factorization the matrix A-/spl lambda/ where A is the ...
Comments