Scalable algorithms for signal reconstruction by leveraging similarity joins

Abstract

Signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an underdetermined system of linear equations that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for SRP do not scale to large problem sizes. In this paper, we propose multiple optimization steps, developing scalable algorithms for the problem. We first propose a dual formulation of the problem and develop the Direct algorithm that is significantly more efficient than the state of the art. Second, we show how adapting database techniques developed for scalable similarity joins provides a significant speedup over Direct, scaling our proposal up to large-scale settings. Third, we describe a number of practical techniques that allow our algorithm to scale to settings of size in the order of a million by a billion. We also adapt our proposal to identify the top-k components of the solved system of linear equations. Finally, we consider the dynamic setting where the inputs to the linear system change and propose efficient algorithms inspired by the database techniques of materialization and reuse. Extensive experiments on real-world and synthetic data confirm the efficiency, effectiveness and scalability of our proposal.

Appendices

Appendix

Theoretical negative result for sparsity of $AA^\mathrm{T}$

In this section, we study an adversarial case that shows the negative result that in theory $t=AA^\mathrm{T}$ can be non-sparse and even thresholding will not significantly help to make them sparse.

Fig. 36

Illustration of chain, the adversarial case for the sparsity of $AA^\mathrm{T}$

We found the negative result in the context of traffic matrix reconstruction. We use the chain graph (Fig. 36), while considering all pairs of nodes as valid SD pairs, as an adversarial case in this section. This provides the upper bound on the number of nonzero elements in $t=AA^\mathrm{T}$:

Theorem 4

$n^2$ is the tight upper bound on the number of nonzero elements in $t=AA^\mathrm{T}$ for SRP.

Proof

We use the chain graph for proving the upper bound. As we discussed in Sect. 5.2, each cell t[i, j] is the number of SD pairs that contain both edges $e_i$ and $e_j$ in their route. In following, we show that every pair of edges in the chain share at least one flow. Note that in the chain, there is a unique path between each pair of nodes, containing the set of edges between the two nodes. Consider the first and last nodes in the chain, i.e., $N_1$ and $N_{r}$ in Fig. 36. The path between these two nodes contains $\{e_1,e_2, \cdots , e_{r-1}\}$, i.e., all edges of the graph. It means for any pair $e_i$ and $e_j$, there exists at least one SD pair that contains both of them in their path. As a result, for the case of chain, all cells of matrix t are nonzero. The existence of a case for which none of the elements in $t=AA^\mathrm{T}$ are zero provides the tight upper bound on the nonzero elements of t as $n^2$.$\square$

The problem in the chain is that the choice of paths between the SD pairs is very limited and all traffic should flow within the same paths. Fortunately, in practice, having a graph structure, there are multiple alternatives for paths, reducing the chance that two random edges share at least a flow. In our experiments in Sect. 7, the number of nonzero values was always below 2% of $AA^\mathrm{T}$ cells.

Next, we show that even thresholding cannot help, increasing the sparsity in the chain. For simplicity of explanations, let the indexing of nodes and edges be as presented in Fig. 36. Note that in this graph, there are $n=r-1$ edges and $m={r \atopwithdelims ()2}$ SD pairs. The first observation is that there is only one pair of edges ($e_1$ and $e_{r-1}$) that only share one flow between them. That is, only the cells $t[1,r-1]$ and $t[r-1,1]$ have the value of 1, and all other cells have larger values. It means with the threshold of $\tau =2$, there are only two cells with values less than $\tau$ in $t = AA^\mathrm{T}$.

Now, consider a pair of edges $e_i$ and $e_j$ where $i\le j$. Looking at Fig. 36, there are exactly i nodes in the left-hand side of $e_i$ and exactly $(r-j)$ nodes in the right-hand side of $e_j$. Note that only the SD pairs with one of their nodes in the left-hand side of $e_i$ and the other in the right-hand side of $e_j$ have both $e_i$ and $e_j$ in their paths. There exist $i(r-j)$ such pairs. Following this, one can calculate the value of t[i, j] as $t[i,j]=t[j,i] = i(r-j)$. Therefore, the number of cells with values smaller than a threshold $\tau$ is equal to two times the number of cases of $i\le j$ such that $i(r-j)< \tau$. For a fixed value $i\in [i,\tau )$:

$$\begin{aligned} i(r-j)\le \tau -1 ~\Rightarrow ~ j \ge r - \frac{\tau -1}{i}~, j< r \end{aligned}$$

This means, for each such value $i\in [i,\tau )$, there are $\frac{\tau -1}{i}$ values of j satisfying the above condition. In other words, for each such value i, there are $2\frac{\tau -1}{i}$ cells in t with values at least equal to $\tau$. Using this, the number of cells with nonzero elements in $t=AA^\mathrm{T}$ is:

$$\begin{aligned} 2 \sum \limits _{i=1}^{\tau -1} \frac{\tau -1}{i} = 2H_{\tau -1}(\tau -1) \end{aligned}$$

(14)

where H is the hyperbolic function.

Equation 14 shows the negative result that for a chain, even thresholding will not make $AA^\mathrm{T}$ sparse. That is because the number of cells with value less that $\tau$ is independent of the size of $AA^\mathrm{T}$. As a result, for a large value of r, the number of nonzero elements in $AA^\mathrm{T}$ almost does not reduce by thresholding.