Reconstruction of multi-view compressed imaging using weighted total variation

Abstract

This paper concentrates on the problem of image reconstruction from compressed sensing (CS) measurements in multi-view compressed imaging systems, where each view is acquired independently by CS technique. In order to take advantage of both the inter-view correlation and the spatial prior information in multi-view image sets, a weighted total variation (TV) regularized model, which combines the TV norm of a target view and the TV norm of the corresponding residual, is proposed. To efficiently solve the weighted TV regularization constrained problem, novel algorithms are presented for both the anisotropy TV and the isotropy TV cases. Given the multi-view CS measurements, a sliding window-based recovery framework is also developed to work with the weighted TV-based reconstruction algorithms and produce high-quality results. We show by experiments that the proposed methods greatly outperform the straight forward reconstruction which applies view by view image reconstruction independently, and also have significant advantages over other benchmark methods.

Appendix

1.1 A Derivation of RA-WATV

Given the weighted anisotropic TV model (13), we first introduce a new variable E _i  = u _i  − p _i and yield

$$\begin{aligned} (\hat{{\bf u}}_{i},\hat{{\bf E}}_{i})&= \mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_{i}} \|\nabla_x{\bf u}_i\|_1+\|\nabla_y{\bf u}_i\|_1+ \beta(\|\nabla_x{\bf E}_i\|_1+\|\nabla_y{\bf E}_i\|_1)\\ &\quad \hbox{s.t.} \quad {\bf y}_i=\boldsymbol{\Upphi}_i{\bf u}_i, \quad {\bf E}_i={\bf u}_i-{\bf p}_{i} \end{aligned}$$

(A1)

Then the model (A1) can be approximated by the following model

$$\begin{aligned} (\hat{{\bf u}}_i,\hat{{\bf E}}_i)&= \mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_i} \|\nabla_x{\bf u}_i\|_1+\|\nabla_y{\bf u}_i\|_1+ \beta(\|\nabla_x{\bf E}_i\|_1+\|\nabla_y{\bf E}_i\|_1)\\ &\quad+\frac{\lambda}{2}(\|{\bf y}_i-\boldsymbol{\Upphi}_i{\bf u}_i\|^2_2+\mu\|{\bf E}_i-({\bf u}_i-{\bf p}_i)\|^2_2) \end{aligned}$$

(A2)

where μ is used to tradeoff between two kinds of approximation error, and λ is used to tradeoff between regularizing functional and approximation error. We introduce (A2) because it is numerically easier to minimize. It is well known that the solution of (A2) converges to that of (A1) as $\lambda\rightarrow\infty$.

After that, we apply the split Bregman method [16] to help solve the problem (A2). Replace $\nabla_{x}{\bf u}_i$ by ${\bf d}_{1x}, \nabla_y{\bf u}_i$ by ${\bf d}_{1y}, \nabla_x{\bf E}_i$ by ${\bf d}_{2x}, \nabla_y{\bf E}_i$ by d _2y , and problem (A2) becomes a new constrained problem

$$\begin{aligned} &(\hat{{\bf u}}_i,\hat{{\bf E}}_i,\hat{{\bf d}}_{1x},\hat{{\bf d}}_{1y},\hat{{\bf d}}_{2x},\hat{{\bf d}}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_i,{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|{\bf d}_{1x}\|_1+\|{\bf d}_{1y}\|_1+\beta(\|{\bf d}_{2x}\|_1+\|{\bf d}_{2y}\|_1)\\ & \qquad +\frac{\lambda}{2}(\|{\bf y}_i-\boldsymbol{\Upphi}_i{\bf u}_i\|^2_2+\mu\|{\bf E}_i-({\bf u}_i-{\bf p}_i)\|^2_2)\\ &\quad \hbox{s.t.}\quad {\bf d}_{1x}=\nabla_x{\bf u}_i,{\bf d}_{1y}=\nabla_y{\bf u}_i, {\bf d}_{2x}=\nabla_x{\bf E}_i,{\bf d}_{2y}=\nabla_y{\bf E}_i \end{aligned}$$

(A3)

Similarly, penalty function terms are added to form the unconstrained problem below

$$\begin{aligned} &(\hat{{\bf u}}_i,\hat{{\bf E}}_i,\hat{{\bf d}}_{1x},\hat{{\bf d}}_{1y},\hat{{\bf d}}_{2x},\hat{{\bf d}}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_i,{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|{\bf d}_{1x}\|_1+\|{\bf d}_{1y}\|_1+\beta(\|{\bf d}_{2x}\|_1+\|{\bf d}_{2y}\|_1)\\ & \qquad+\frac{\lambda}{2}(\|{\bf y}_i-{\boldsymbol{\Upphi}}_i{\bf u}_i\|^2_2 +\mu\|{\bf E}_i-({\bf u}_i-{\bf p}_i)\|^2_2)\\ &\qquad+\frac{\gamma}{2}(\|{\bf d}_{1x}-\nabla_x{\bf u}_i\|^2_2+\|{\bf d}_{1y}-\nabla_y{\bf u}_i\|^2_2)\\ & \qquad+\frac{\gamma}{2}\alpha(\|{\bf d}_{2x}-\nabla_x{\bf E}_i\|^2_2+\|{\bf d}_{2y}-\nabla_y{\bf E}_i\|^2_2) \end{aligned}$$

(A4)

where γ and α are tradeoff parameters, which need to be tuned suitably. Finally, strictly enforce the constraints by applying the Bregman iteration [42] to get the solution for the (k + 1)th iteration

$$\begin{aligned} &({\bf u}^{k+1}_i,{\bf E}^{k+1}_i,{\bf d}^{k+1}_{1x},{\bf d}^{k+1}_{1y},{\bf d}^{k+1}_{2x},{\bf d}^{k+1}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_i,{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|{\bf d}_{1x}\|_1+\|{\bf d}_{1y}\|_1+\beta(\|{\bf d}_{2x}\|_1+\|{\bf d}_{2y}\|_1)\\ &\qquad+\frac{\lambda}{2}(\|{\bf y}_i-\boldsymbol{\Upphi}_i{\bf u}_i\|^2_2+\mu\|{\bf E}_i-({\bf u}_i-{\bf p}_i)\|^2_2)\\ &\qquad+\frac{\gamma}{2}(\|{\bf d}_{1x}-\nabla_x{\bf u}_i-{\bf b}^k_{1x}\|^2_2+\|{\bf d}_{1y}-\nabla_y{\bf u}_i-{\bf b}^k_{1y}\|^2_2)\\ &\qquad+\frac{\gamma}{2}\alpha(\|{\bf d}_{2x}-\nabla_x{\bf E}_i-{\bf b}^k_{2x}\|^2_2+\|{\bf d}_{2y}-\nabla_y{\bf E}_i-{\bf b}^k_{2y}\|^2_2) \end{aligned}$$

(A5)

where the values of b ^k_1x , b ^k_1y , b ^k_2x and b ^k_2y are chosen through Bregman iteration method. Since the variables in the minimization problem (A5) decouple from each other, the target problem can be separated into many sub-problems, two of which are listed below

$$\begin{aligned} {\bf u}^{k+1}_i &=\mathop{\hbox{argmin}}\limits_{{\bf u}_i} \frac{\lambda}{2}(\|{\bf y}_i-{\boldsymbol{\Upphi}}_i{\bf u}_i\|^2_2+\mu\|{\bf E}^k_i-({\bf u}_i-{\bf p}_i)\|^2_2)\\ &+\frac{\gamma}{2}(\|{\bf d}^k_{1x}-\nabla_x{\bf u}_i-{\bf b}^k_{1x}\|^2_2 +\|{\bf d}^k_{1y}-\nabla_y{\bf u}_i-{\bf b}^k_{1y}\|^2_2) \end{aligned}$$

(A6)

$$\begin{aligned} {\bf E}^{k+1}_i&=\mathop{\hbox{argmin}}\limits_{{\bf E}_i} \frac{\lambda}{2}(\mu\|{\bf E}_i-({\bf u}^{k+1}_i-{\bf p}_i)\|^2_2)\\ &+\frac{\gamma}{2}\alpha(\|{\bf d}^k_{2x}-\nabla_x{\bf E}_i-{\bf b}^k_{2x}\|^2_2 +\|{\bf d}^k_{2y}-\nabla_y{\bf E}_i-{\bf b}^k_{2y}\|^2_2) \end{aligned}$$

(A7)

Note that after sub-problem (A6) is solved, the updated u ^k+1 _i can be used in sub-problem (A7). Since object functions of (A6) and (A7) are quadratic, we can find their optimality conditions (15) and (16), respectively. Using CGS method [30], we can easily get updating solution for u ^k+1 and E ^k+1.

For the remaining sub-problem

$$\begin{aligned} &({\bf d}^{k+1}_{1x},{\bf d}^{k+1}_{1y},{\bf d}^{k+1}_{2x},{\bf d}^{k+1}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|{\bf d}_{1x}\|_1+\|{\bf d}_{1y}\|_1+\beta(\|{\bf d}_{2x}\|_1+\|{\bf d}_{2y}\|_1)\\ &\qquad+\frac{\gamma}{2}(\|{\bf d}_{1x}-\nabla_x{\bf u}^{k+1}_i-{\bf b}^k_{1x}\|^2_2 +\|{\bf d}_{1y}-\nabla_y{\bf u}^{k+1}_i-{\bf b}^k_{1y}\|^2_2)\\ &\qquad+\frac{\gamma}{2}\alpha(\|{\bf d}_{2x}-\nabla_x{\bf E}^{k+1}_i-{\bf b}^k_{2x}\|^2_2 +\|{\bf d}_{2y}-\nabla_y{\bf E}^{k+1}_i-{\bf b}^k_{2y}\|^2_2) \end{aligned}$$

(A8)

It can be divided into 4 typical l ₂-l ₁ sub-problems with respect to d _1x , d _1y , d _2x and d _2y , each of which can be solved by shrinkage operation explained in Sect. 3.3.

1.2 B Derivation of RA-WITV

For the weighted isotropic TV model (14), similarly, it is converted to

$$\begin{aligned} &({\bf u}^{k+1}_i,{\bf E}^{k+1}_i,{\bf d}^{k+1}_{1x},{\bf d}^{k+1}_{1y},{\bf d}^{k+1}_{2x},{\bf d}^{k+1}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf u}_i,{\bf E}_i,{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|({\bf d}_{1x},{\bf d}_{1y})\|_2+\beta(\|({\bf d}_{2x},{\bf d}_{2y})\|_2)\\ &\qquad+\frac{\lambda}{2}(\|{\bf y}_i-{\boldsymbol{\Upphi}}_i{\bf u}_i\|^2_2+\mu\|{\bf E}_i-({\bf u}_i-{\bf p}_i)\|^2_2)\\ &\qquad+\frac{\gamma}{2}(\|{\bf d}_{1x}-\nabla_x{\bf u}_i-{\bf b}^k_{1x}\|^2_2+\|{\bf d}_{1y} -\nabla_y{\bf u}_i-{\bf b}^k_{1y}\|^2_2)\\ &\qquad+\frac{\gamma}{2}\alpha(\|{\bf d}_{2x}-\nabla_x{\bf E}_i-{\bf b}^k_{2x}\|^2_2+\|{\bf d}_{2y} -\nabla_y{\bf E}_i-{\bf b}^k_{2y}\|^2_2) \end{aligned}$$

(B1)

where $\|({\bf d}_{x},{\bf d}_{y})\|_2=\sum_{j}{\sqrt{d^2_{x,j}+d^2_{y,j}}}; d_{x,j}$ and d _y,j stand for the jth element in vector d _x and d _y , respectively.

Problem (B1) is also expected to be divided into many disconnected sub-problems and solved iteratively. Obviously, sub-problems for updating u ^k+1 _i and E ^k+1 _i still have the same forms as (A6) and (A7). However, the d _1x and d _1y variables do not decouple as they did in the anisotropic case, and neither are the d _2x and d _2y variables. In order to apply the iterative minimization procedure to this problem, we must solve the following sub-problem instead of problem (A8)

$$\begin{aligned} &({\bf d}^{k+1}_{1x},{\bf d}^{k+1}_{1y},{\bf d}^{k+1}_{2x},{\bf d}^{k+1}_{2y})\\ &\quad=\mathop{\hbox{argmin}}\limits_{{\bf d}_{1x},{\bf d}_{1y},{\bf d}_{2x},{\bf d}_{2y}} \|({\bf d}_{1x},{\bf d}_{1y})\|_2+\beta(\|({\bf d}_{2x},{\bf d}_{2y})\|_2)\\ &\qquad+\frac{\gamma}{2}(\|{\bf d}_{1x}-\nabla_x{\bf u}^{k+1}_i-{\bf b}^k_{1x}\|^2_2 +\|{\bf d}_{1y}-\nabla_y{\bf u}^{k+1}_i-{\bf b}^k_{1y}\|^2_2)\\ &\qquad+\frac{\gamma}{2}\alpha(\|{\bf d}_{2x}-\nabla_x{\bf E}^{k+1}_i -{\bf b}^k_{2x}\|^2_2+\|{\bf d}_{2y}-\nabla_y{\bf E}^{k+1}_i-{\bf b}^k_{2y}\|^2_2) \end{aligned}$$

(B2)

Despite we cannot treat the d _1x , d _1y , d _2x , d _2y variables as we did in the anisotropic case, we are still able to explicitly solve the minimization problem for updating these variables using a generalized shrinkage formula [38]. Thus, different from step 2 of Algorithm 1, for RA-WITV, the jth element in d ^k_1x , d ^k+1_1y , d ^k+1_2x , d ^k+1_2y should be updated by

$$\begin{aligned} ({\bf d}^{k+1}_{1x})_j&=\max(({\bf s}^k_1)_j-1/\gamma,0)\cdot\frac{(\nabla_x{\bf u}^{k+1}_i)_j +({\bf b}^k_{1x})_j}{({\bf s}^k_1)_j}\\ ({\bf d}^{k+1}_{1y})_j&=\max(({\bf s}^k_1)_j-1/\gamma,0)\cdot\frac{(\nabla_y{\bf u}^{k+1}_i)_j +({\bf b}^k_{1y})_j}{({\bf s}^k_1)_j}\\ ({\bf d}^{k+1}_{2x})_j&=\max(({\bf s}^k_2)_j-\beta/(\alpha\gamma),0)\cdot\frac{(\nabla_x{\bf E}^{k+1}_i)_j+({\bf b}^k_{2x})_j}{({\bf s}^k_2)_j}\\ ({\bf d}^{k+1}_{2y})_j&=\max(({\bf s}^k_2)_j-\beta/(\alpha\gamma),0)\cdot\frac{(\nabla_y{\bf E}^{k+1}_i)_j+({\bf b}^k_{2y})_j}{({\bf s}^k_2)_j} \end{aligned}$$

(B3)

where

$$\begin{aligned} ({\bf s}^k_1)_j&=\sqrt{((\nabla_x{\bf u}^{k+1}_i)_j+({\bf b}^k_{1x})_j)^2+((\nabla_y{\bf u}^{k+1}_i)_j+({\bf b}^k_{1y})_j)^2}\\ ({\bf s}^k_2)_j&=\sqrt{((\nabla_x{\bf E}^{k+1}_i)_j+({\bf b}^k_{2x})_j)^2+((\nabla_y{\bf E}^{k+1}_i)_j+({\bf b}^k_{2y})_j)^2} \end{aligned}$$

(B4)