Sparse signal recovery via infimal convolution based penalty☆
Introduction
In recent years, sparse recovery problems have drawn people’s attention in many applications such as compressive sensing (CS) [1], [2], [3], [4], machine learning [5], [6] and image processing [7], [8], [9], [10]. These problems are usually formulated as where is the unknown signal that is sparse or can be sparsely represented on an appropriate basis, is the loss function related to data fidelity term, is the regularized function to penalize the sparsity of , and is the penalty parameter to balance the regularization and data fidelity.
For a linear sampling system , where is the observation noise or error, the loss function is often selected to be least-squares (LS) of the residuals as based on the assumption of Gaussian distributed noise condition, or the least-absolute (LA) of the residuals as based on the impulsive noise assumption such as Cauchy distribution [1], [11].
On the other hand, the penalty function should be selected as the -norm intuitively, which represents the number of nonzero elements in . However, minimizing the -norm is a NP-hard problem. A frequently used method is choosing its -norm convex approximation, i.e., to replace it [12]. This convex relaxation model has been widely used in many different fields, such as communications [13], synthetic aperture radar (SAR) images processing [6], direction of arrival (DOA) estimation [14] and magnetic resonance imaging (MRI) [7]. It has been proved that the -sparse signal can be recovered by the model under some assumptions of the operator , such as the restricted isometry property (RIP) of if the operator is a sensing matrix [15]. Although the -norm regularization can induce sparsity most effectively among convex regularizers because of its shrinkage operator, it tends to underestimate high-amplitude components of as it uniformly penalizes the amplitude, unlike that all nonzero entries have the same punishment in the original -norm. This property of -norm may lead to reconstruction failures with the least measurements [16] and sometimes brings undesirable blocky images [8].
Recently, there are some works which do not approximate the -norm but directly deal with the -norm or the -sparse constrained problem, such as the iterative hard thresholding (IHT) algorithm [17] and its variants and acceleration versions: accelerated IHT (AIHT) [18], proximal IHT (PIHT) [19], extrapolated proximal IHT (EPIHT) [20] and accelerated proximal IHT [21]. Another idea is to transform the model into an equivalent minimization problem, such as the difference of two convex function , where denotes the sum of top- elements of in absolute value [22]. Recently, this model has been extended to more general situations, such as the partial regularization [23] with representing the th-largest elements in , and the -difference regularization [24] with representing the best term approximation to , where and are some regularizers satisfying certain assumptions.
Meanwhile, some researchers focus on finding the non-convex regularizers, which can approximate the -norm better and achieve effective performance, such as the (quasi)-norm with [25], capped -norm [26], the minimax-concave penalty (MCP) [27] and the difference of the and -norms () [28], [29], log-sum penalty (LSP) [30], smoothly clipped absolute deviation (SCAD) [9], correntropy induced metric (CIM) penalty [31], [32]. Among them, the has achieved impressive results that its almost sure convergence to a global minimum with the help of a simulated annealing (SA) procedure and good performance in the condition of highly coherent matrix.
The main challenge faced by those non-convex regularizations induced minimization problems is how to solve them effectively. Many iterative algorithms are investigated by researchers, such as the difference of convex algorithm (DCA) [33], [34] and its accelerate versions: Boosted DCA (BDCA) [35] and proximal DCA with extrapolation (pDCAe) [36], General Iterative Shrinkage and Thresholding (GIST) [37], alternating direction method of multipliers (ADMM) [10], split Bregman iteration (SBI) [2], and nonmonotone accelerated proximal gradient (nmAPG) [38], which is an extension of the accelerated proximal gradient (APG) [39].
Underestimating high-amplitude components is one of the most frequently stated problems of -norm regularization. To deal with this problem, first, we propose a non-convex penalty function to retain the advantages of convex norm and non-convex norm in this paper, which can be flexibly adjusted between and . This means that it can induce the sparsity effectively for the low-amplitude components and relieve underestimating high-amplitude components. At the same time, it can keep the objective function convex under certain conditions by using the infimal convolution.
Second, to solve the non-convex penalty regularized minimization problem, we employ the DCA algorithm and forward–backward splitting (FBS) algorithm, respectively. We prove that any cluster point of the sequence generated by these two algorithms converges to a stationary point. In addition, we also derive a closed-form solutions for the proximal gradient operator of FBS, which can accelerate the FBS.
Finally, we discuss some properties of the proposed non-convex penalty, which can be easily extended to other sparse recovery problems. We also evaluate the effectiveness of the proposed algorithms via numerical experiments.
The overall structure of the study takes the form of five sections, including this introductory section. Section Two begins by laying out the definition of infimal convolution based penalty (ICP), and looks at the theoretical properties of ICP. The third section is concerned with the iterative algorithms used for solving ICP based non-convex problem. Section four presents the numerical results. In the end, we provide our conclusion in Section five.
Here, we define our notation. We define the -norm of the vector as . Especially, we define , and -norms of as , and , respectively. For any given matrix , represents the transpose of , represents the maximum eigenvalue of , is the submatrix of with column indices and being the cardinality of . means that the matrix is positive semidefinite. represents an identity matrix. denotes the inner product. The set of proper lower semicontinuous convex functions from to is defined as .
Section snippets
Infimal convolution based penalty
We first recall the definition of infimal convolution. For two functions and from to , the infimal convolution [40] is given by Instead of the frequently used , we propose a new penalty function as follows.
Definition 1 Let , . We definite the infimal convolution based penalty as where is the -norm infimal convolution defined as
Property 1 The defined function satisfies the following properties. (a)
Two iterative algorithms for ICP minimization problem
In this section, we employ two iterative frameworks for solving the unconstrained ICP based non-convex problem under the assumption that observation noise obeys Gaussian distribution. We consider the following minimization problem where is the measurement matrix and is the measurement data.
Numerical experiments
In this section, we present numerical experiments to demonstrate the efficiency of the ICP. We apply six methods in comparison with the proposed algorithm: (1) the -norm regularization based ADMM-lasso [5]; (2) the semismooth Newton augmented Lagrangian (SSNAL) method [45] for LASSO problem ( http://www.math.nus.edu.sg/ mattohkc/SuiteLasso.html); (3) the iterative -shrinkage (IPS) algorithm [3] with , which uses the -shrinkage mapping as ; (4) the
Conclusions
In this paper, we have proposed an infimal convolution based penalty function for sparse signal recovery and employed the two iterative methods to solve the non-convex optimization problem: one of them uses the DCA framework with the ADMM solving the subproblem, the other one employs the FBS with the closed-form proximal operator. The convergence of the proposed algorithm is proved. The experimental results demonstrate the effectiveness of the proposed method by comparing with some other
CRediT authorship contribution statement
Lin Lei: Developed the idea, Writing - original draft. Yuli Sun: Developed the idea, Formal analysis, Writing - original draft. Xiao Li: Formal analysis.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The authors would like to thank the editors and anonymous reviewers for their careful reading of an earlier version of this article and constructive suggestions that improved the presentation of this work.
References (48)
- et al.
Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization
Signal Process.
(2014) - et al.
Computed tomography image reconstruction from few views via Log-norm total variation minimization
Digit. Signal Process.
(2019) - et al.
Smoothly clipped absolute deviation (SCAD) regularization for compressed sensing MRI using an augmented Lagrangian scheme
Magn. Reson. Imag.
(2013) - et al.
A robust recovery algorithm with smoothing strategies
Neurocomputing
(2020) - et al.
Sparse Bayesian learning for off-grid DOA estimation with nested arrays
Digit. Signal Process.
(2018) The restricted isometry property and its implications for compressed sensing
CR Math.
(2008)- et al.
Iterative hard thresholding for compressed sensing
Appl. Comput. Harmon. Anal.
(2009) Accelerated iterative hard thresholding
Signal Process.
(2012)Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-gaussian environments
J. Franklin Inst.
(2015)- et al.
Alternating direction algorithms for L1-problems in compressive sensing
SIAM J. Sci. Comput.
(2011)
Compressed sensing recovery via nonconvex shrinkage penalties
Inverse Problems
Image compressive sensing recovery via collaborative sparsity
IEEE J. Emerg. Sel. Topics Power Electron.
Distributed optimization and statistical learning via the alternating direction method of multipliers
Found. Trends Mach. Learn.
Classification via sparse representation of steerable wavelet frames on grassmann manifold: Application to target recognition in SAR image
IEEE Trans. Image Process.
Compressed sensing MRI
IEEE Signal Proc. Mag.
Efficient and robust recovery of sparse signal and image using generalized nonconvex regularization
IEEE Trans. Comput. Imag.
Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
IEEE Trans. Inform. Theory
Application of compressive sensing to sparse channel estimation
IEEE Commun. Mag.
Restricted isometry properties and nonconvex compressive sensing
Inverse Problems
Iterative hard thresholding methods for L0 regularized convex cone programming
Math. Program.
Image restoration by minimizing zero norm of wavelet frame coefficients
Inverse Problems
An accelerated proximal iterative hard thresholding method for L0 minimization
DC formulations and algorithms for sparse optimization problems
Math. Program.
Sparse recovery via partial regularization: Models, theory, and algorithms
Math. Oper. Res.
Cited by (2)
Bearing fault diagnosis via generalized logarithm sparse regularization
2022, Mechanical Systems and Signal ProcessingCitation Excerpt :However, in extreme cases, the energy of noises might be great and the signal-to-noise ratio(SNR) may be small, these methods are difficult to reconstruct fault component from the monitoring signal accurately and affect heavily by the noise. Over the past decades, a lot of attention has been attracted to sparse representation technique in the signal processing field [11–14]. Because the repetitive transients can be approximated by few dictionary atoms while the noise cannot, so that the sparse representation method is able to decompose the monitoring vibration signal to a vector called sparse coefficient, whose elements are mostly equal to zero.
Gearbox fault diagnosis based on generalized multivariate logarithmic regularization
2023, Measurement Science and Technology
- 1
These authors contributed equally.