Reordering sparse matrices into block-diagonal column-overlapped form

Highlights

• Reorders sparse matrices into block-diagonal column-overlapped (BDCO) form.

• Minimizes the total number of coupling columns between diagonal blocks.

• Balances the number of nonzeros in diagonal blocks.

• Proposes a hypergraph partitioning model to solve the BDCO reordering problem.

• Uses recursive bipartitioning and fixed vertices.

• Experimental results validate the effectiveness of the proposed model.

Abstract

Many scientific and engineering applications necessitate computing the minimum norm solution of a sparse underdetermined linear system of equations. The minimum 2-norm solution of such systems can be obtained by a recent parallel algorithm, whose numerical effectiveness and parallel scalability are validated in both shared- and distributed-memory architectures. This parallel algorithm assumes the coefficient matrix in a block-diagonal column-overlapped (BDCO) form, which is a variant of the block-diagonal form where the successive diagonal blocks may overlap along their columns. The total overlap size of the BDCO form is an important metric in the performance of the subject parallel algorithm since it determines the size of the reduced system, solution of which is a bottleneck operation in the parallel algorithm. In this work, we propose a hypergraph partitioning model for reordering sparse matrices into BDCO form with the objective of minimizing the total overlap size and the constraint of maintaining balance on the number of nonzeros of the diagonal blocks. Our model makes use of existing partitioning tools that support fixed vertices in the recursive bipartitioning paradigm. Experimental results validate the use of our model as it achieves small overlap size and balanced diagonal blocks.

Introduction

Many scientific and engineering applications [6], [12], [19], [22], [25], [27] require computing the minimum norm solution of an underdetermined system of equations of the form

$$Ax=f,$$

where $A$ is an $m\times n$ sparse matrix and $m<n$ [18]. A common approach for obtaining the minimum 2-norm solution of an underdetermined linear least squares problem is the use of a QR factorization in a direct method. Various packages such as SuiteSparseQR [13], [14], HSL MA49 [2], and qr_mumps [7] provide efficient parallel and sequential implementations for the general sparse QR algorithm.

One recent approach [24] for obtaining the minimum 2-norm solution of an underdetermined linear least squares problem is the use of the Balance scheme [17], [21], [23], which is an effective and efficient parallel algorithm originally proposed for an ill-conditioned banded linear system of equations. This parallel algorithm [24] can also be considered as an extension of the general sparse QR factorization to distributed-memory systems for obtaining the minimum 2-norm solution. In this parallel algorithm, the coefficient matrix is assumed to be in block-diagonal column-overlapped (BDCO) form with $K$ diagonal blocks, which is shown in Fig. 1. As seen in the figure, the $k$th diagonal block $E_k$ of the BDCO form is given by

$$E_k=\left[C_k{A}_k{B}_k\right],$$

for $k=2,3,\dots ,K-1$, whereas the first and last diagonal blocks $E_1$ and $E_K$ are respectively given by

$$E_1=\left[A_1{B}_1\right]\text{and}{E}_K=\left[C_K{A}_K\right].$$

Here, successive diagonal blocks $E_{k-1}$ and $E_k$ overlap along the columns of their submatrices $B_{k-1}$ and $C_k$. Columns of the overlapping submatrices are referred to as coupling columns.

The parallel algorithm [24] exploits the BDCO form so that each of $K$ processors independently solves a small linear least squares problem of the form

$$E_k{z}_k=f_k,$$

where

$$z_1=\left[\begin{array}{c}\hfill x_1\hfill \\ \hfill e_1\hfill \end{array}\right],z_k=\left[\begin{array}{c}\hfill {\hat{e}}_{k-1}\hfill \\ \hfill x_k\hfill \\ \hfill e_k\hfill \end{array}\right]\text{ for }k=2,3,\dots ,K-1,\text{ and }{z}_K=\left[\begin{array}{c}\hfill {\hat{e}}_{K-1}\hfill \\ \hfill x_K\hfill \end{array}\right].$$

This step is performed in parallel without any communication. For $x$ to be a solution, $e_k$ and ${\hat{e}}_k$ should be equal for $k=1,2,\dots ,K-1$. This is ensured by a sequential step in which a small system called reduced system is solved. The size of the reduced system is determined by the total overlap size, i.e., the total number of coupling columns. When compared to the state-of-the-art parallel QR implementations, this approach is reported to achieve better scalability on both shared- and distributed-memory architectures [24].

In this paper, our aim is to find two permutations $P$ and $Q$ such that $PAQ$ is in a $K$-way BDCO form. We refer to this permutation problem as the BDCO problem. In the BDCO problem, the objective is to minimize the total overlap size in $PAQ$ since the performance of the above-mentioned parallel algorithm considerably deteriorates with increasing total overlap size, as reported in [24]. The constraint of the BDCO problem is to maintain balance on the number of nonzeros in the diagonal blocks of $PAQ$ to ensure balanced workload for processors.

To the best of our knowledge, literature lacks an algorithm for solving the BDCO problem. However, there are efforts for solving problems that resemble the BDCO problem. In [5] and [26], the problems of reordering sparse matrices into singly-bordered block diagonal (SBBD) form and separated block diagonal (SBD) form were addressed, respectively. In the SBBD form, all coupling rows are reordered to a border at the bottom or all coupling columns are reordered to a border on the right. In the SBD form, coupling rows are reordered in between two parts as they are encountered during the recursive bipartitioning steps. The SBBD and SBD forms differ from the BDCO form as they allow coupling rows/columns to span more than two diagonal blocks, which are not necessarily consecutive. In [1], the problem of reordering sparse square matrices into block diagonal form with overlap (BDO form) was addressed. The differences between the BDCO and BDO forms are two-fold: (i) the BDO form is obtained on structurally symmetric matrices via symmetric row/column permutation, whereas the BDCO form is obtained on non-square matrices via row and column permutations that are different from each other, (ii) consecutive diagonal blocks overlap along both rows and columns in the BDO form, whereas they overlap along only columns in the BDCO form.

In order to solve the BDCO problem, we propose the HP${}_{\mathrm{BDCO}}$ algorithm, which utilizes the column-net hypergraph model [8] of the given coefficient matrix $A$. First, we investigate the feasibility of the $K$-way BDCO permutation for matrix $A$ by considering the vertex-to-vertex adjacency topology of the column-net hypergraph of $A$. If the respective permutation is feasible, then the corresponding hypergraph is partitioned by the proposed HP${}_{\mathrm{BDCO}}$ algorithm so that the partitioning objective corresponds to minimizing the total overlap size, whereas the partitioning constraint corresponds to maintaining balance on the number of nonzeros of the diagonal blocks. The HP${}_{\mathrm{BDCO}}$ algorithm utilizes fixed vertices within the recursive bipartitioning paradigm in order to ensure that only the successive diagonal blocks may share columns. The proposed algorithm is flexible in the sense that any hypergraph partitioning tool that supports fixed vertices can be used in the implementation. The effectiveness of the proposed HP${}_{\mathrm{BDCO}}$ algorithm is validated by reordering a wide range of matrices into BDCO form with small overlap size and balanced diagonal blocks as well as by running the parallel code [24] on the reordered matrices.

The rest of the paper is organized as follows. Section 2 gives background on hypergraph partitioning. The proposed HP${}_{\mathrm{BDCO}}$ algorithm is given in Section 3. Section 4 gives experimental results and Section 5 concludes.