A multilayer FOCUSS approach for sparse representation

Abstract

Focal Underdetermined System Solver (FOCUSS) is a powerful method for sparse representation, in which the Lp-norm like cost function is very often used. However, this cost function is not only nondifferentiable but also can be very ill-conditioned in some situations. The local minima problem of FOCUSS is discussed in this paper. Moreover, to solve this problem, we first extend the Lp-norm like cost function to its corresponding Lp-approximation. After this, we analyze the nonconvexity of the new cost function, which results in that FOCUSS algorithm gets stuck in the local minima in many situations, especially when the hidden sources are not very sparse. To reduce the number of the local minima, a multilayer FOCUSS is developed in this paper. Comparing with the conventional FOCUSS, the experiments inclusing MRI reconstruction demonstrate that multilayer FOCUSS can significantly improve the performance. Even for some very challenging cases, where the conventional FOCUSS fails, multilayer FOCUSS still works well.

Appendix: Derivation of robust FOCUSS

To solve optimization problem (6), here we similarly use the standard method of Lagrange multipliers to derive the iterative formula (7) for robust FOCUSS. Define the Lagrangian function $L({\varvec{s}},{\varvec{\lambda }})$ as

$$\begin{aligned} L({\varvec{s}},{\varvec{\lambda }})=J({\varvec{s}})+{\varvec{\lambda }}^{T}({\varvec{A}s}-{\varvec{x}}). \end{aligned}$$

(11)

A necessary condition for a minimizing solution sto exist is that (${\varvec{s}}^{*},{\varvec{\lambda }}^{*})$ to be a stationary point of the Lagrangian function $L({\varvec{s}},{\varvec{\lambda }})$. Thus, we have

$$\begin{aligned} \left\{ {{\begin{array}{ll} {\frac{\partial L({\varvec{s}}^{*},{\varvec{\lambda }}^{*})}{\partial {\varvec{s}}}=\frac{\partial J({\varvec{s}}^{*})}{\partial {\varvec{s}}}+{\varvec{A}}^{T}\cdot {\varvec{\lambda }}^{*}=0} \\ {\frac{\partial L({\varvec{s}}^{*},{\varvec{\lambda }}^{*})}{\partial {\varvec{\lambda }}}={\varvec{A}s}-{\varvec{x}}=0} \\ \end{array} }} \right. \end{aligned}$$

(12)

where the partial derivative of $J({\varvec{s}}^{*})$ with respect to $s_i^*$ is

$$\begin{aligned} \frac{\partial J({\varvec{s}}^{*})}{\partial s_i }=\left| p \right| \cdot s_i^*\cdot \left( {\sqrt{\left( {s_i^*} \right) ^{2}+\delta }} \right) ^{p-2},{}{}{}{}i=1,\ldots ,n. \end{aligned}$$

(13)

As in [5, 14], the expression (13) can be re-written as

$$\begin{aligned} \frac{\partial J({\varvec{s}}^{*})}{\partial s_i }=2\left| p \right| \cdot {\varvec{\Pi }}({\varvec{s}}^{*})\cdot {\varvec{s}}^{*}, \end{aligned}$$

(14)

where ${\varvec{\Pi }}({\varvec{s}}^{*})\!=\!diag\left\{ \left[ {\sqrt{(s_1^*)^{2}+\delta }} \right] ^{2-p},\left[ {\sqrt{(s_1^*)^{2}+\delta }} \right] ^{2-p}\!,\right. \cdots ,\left. \left[ {\sqrt{(s_n^*)^{2}+\delta }} \right] ^{2-p} \right\}$. From (12) and (14), we know that the stationary point (${\varvec{s}}^{*},{\varvec{\lambda }}^{*})$ satisfies the following equation sets

$$\begin{aligned} \left\{ {{\begin{array}{ll} {\frac{\partial L({\varvec{s}}^{*},{\varvec{\lambda }}^{*})}{\partial {\varvec{s}}}=2\left| p \right| \cdot {\varvec{\Pi }}({\varvec{s}}^{*})\cdot {\varvec{s}}^{*}+{\varvec{A}}^{T}\cdot {\varvec{\lambda }}^{*}=0} \\ {\frac{\partial L({\varvec{s}}^{*},{\varvec{\lambda }}^{*})}{\partial {\varvec{\lambda }}}={\varvec{A}s}-{\varvec{x}}=0} \\ \end{array} }} \right. \end{aligned}$$

(15)

From (15), carrying out some similar manipulations to [5, 14], we can derive

$$\begin{aligned} {\varvec{s}}^{*}={\varvec{\Pi }}^{-1}({\varvec{s}}^{*})\cdot {\varvec{A}}^{T}\cdot \left[ {{\varvec{A}}\cdot {\varvec{\Pi }}^{-1}({\varvec{s}}^{*})\cdot {\varvec{A}}^{T}} \right] ^{-1}{\varvec{x}}, \end{aligned}$$

which leads to the following iterative formula for solving the optimization problem (6):

$$\begin{aligned} {\varvec{s}}^{(k+1)}={\varvec{\Pi }}^{-1}({\varvec{s}}^{(k)})\cdot {\varvec{A}}^{T}\cdot \left[ {{\varvec{A}}\cdot {\varvec{\Pi }}^{-1}({\varvec{s}}^{(k)})\cdot {\varvec{A}}^{T}} \right] ^{-1}{\varvec{x}}. \end{aligned}$$