Elsevier

Neurocomputing

Volume 458, 11 October 2021, Pages 112-126
Neurocomputing

Manifold constrained joint sparse learning via non-convex regularization

https://doi.org/10.1016/j.neucom.2021.06.008Get rights and content

Abstract

The traditional robust principal component analysis (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications in computer vision. However, the reconstructed image experiences serious interference by Gaussian noise, resulting in the degradation of image quality during the denoising process. Thus, a novel manifold constrained joint sparse learning (MCJSL) via non-convex regularization approach is proposed in this paper. Morelly, the manifold constraint is adopted to preserve the local geometric structures and the non-convex joint sparsity is introduced to capture the global row-wise sparse structures. To solve MCJSL, an efficient optimization algorithm using the manifold alternating direction method of multipliers (MADMM) is designed with closed-form solutions and it achieves a fast and convergent procedure. Moreover, the convergence is analyzed mathematically and numerically. Comparisons among the proposed MCJSL and some state-of-the-art solvers, on several accessible datasets, are presented to demonstrate its superiority in image denoising and background subtraction. The results indicate the importance to incorporate the manifold learning and non-convex joint sparse regularization into a general RPCA framework.

Introduction

Robust principal component analysis (RPCA) is a popular topic in computer vision, which has been extensively applied in the fields such as image denoising and background subtraction [1], [2], [3], [4], [5]. Generally speaking, RPCA intends to discover a low-rank matrix to approximate the original noisy data well. For example, in background subtraction, the surveillance video can be decomposed into the sum of a background component and a foreground component. Since the background component has relatively static small changes over a period, it can be described as a low-rank matrix [6]. During the past decades, a large number of RPCA-related theories, algorithms, and applications have been proposed [7], [8], [9], [10], [11], [12].

Specifically, given an original noisy image D, RPCA intends to estimate the low-rank matrix L and sparse matrix S, with D=L+S. In the optimization of RPCA, Cai et al. [13] utilized the nuclear norm to estimate the low-rank matrix, which is called nuclear norm minimization (NNM). By applying a singular value thresholding, the original data matrix can be reconstructed effectively under certain conditions. However, NNM applies the same thresholding to shrink all the singular values, which does not consider the fact that the image information is mainly concentrated on the larger values. Therefore, the details of the image will become too smooth and blurred. To overcome this shortcoming, Gu et al. [14] proposed weighted nuclear norm minimization (WNNM), which integrates the weighted technique with non-negative matrix factorization (NMF). They proved that WNNM can obtain better approximation via numerical experiments in image denoising and background subtraction. It would not be exaggerating to express that WNNM has become one of the most prevalent methods to resolve the RPCA. Due to the rapid developments in non-convex optimization, one can further improve the performance of NNM-based RPCA by adding non-convex regularization terms [15], [16], [17], [18], [19]. The fundamental reason for this is the L1 norm and nuclear norm of NNM-based RPCA are convex but with loose relaxations, which can not exploit the latent structures. Therefore, a tighter approximation can be pursued. Hu et al. [15] truncated the nuclear norm by removing small residuals, which enhances the robustness to noises. Quach et al. [16] replaced L1 norm with Lp norm (0<p<1) and yielded a more sparse solution. Parekh et al. [17] presented a parameterized non-convex substitution to estimate the low-rank matrix, which can not only solve the low-rank minimization problem well but also achieve better results in image denoising. Chen [18] formulated a non-convex and non-separable regularization from a Bayesian perspective, which demonstrates encouraging performance in hyperspectral images. Wen et al. [19] considered a generalized nonconvex regularization, including Lp norm and other non-convex functions such as smoothly clipped absolute deviation and minimax concave penalty. Wang et al. [20] proposed to add non-convex penalty function and a weighted version to RPCA-based model, which is a better low-rank approximation for image denosing.

Unfortunately, these aforementioned RPCA-based methods may not be able to extract valuable features in high-dimensional settings. From the aspect of image denoising, each row is related to one specific feature. Hence, it is natural to consider joint sparsity (i.e., L2,1 norm) rather than L1 norm. In order to facilitate better interpretations of selected features, joint sparsity is constructed by imposing L2,1 norm constraints on variables [21], [22]. It is illustrated that the joint sparsity can force the features not included in the projected space to be zero rows, which reduces the influence of outliers and abandons invalid features. Consequently, RPCA based on joint sparsity shares both the advantages of sparsity methods over dimension reduction methods, and over other RPCA-based methods.

Actually, if the samples in the high-dimensional space are distributed on the low-dimensional manifolds, it is easy to capture the low-level structures through RPCA-based methods. However, this assumption can not be effectively guaranteed in real-world applications. Inspired by recent advances of manifold learning, manifold constraints can be considered to maintain local geometric structures embedded in the high-dimensional space [23]. Enforcing manifold constraints always provides significant improvements in image denoising when one maps the original data onto a well-designed space [24]. Recently, Liu et al. [25] considered the manifold constrained RPCA, which takes advantages of manifold learning and RPCA. Compared with other kinds of RPCA, manifold constrained RPCA can both improve the performance of image reconstruction and reduce the running time. However, only manifold prior of the low-rank component is discussed, which leads to limited feature selection. Therefore, there is a need to construct a variant of RPCA that can extract meaningful features and preserve local geometric structures.

Motivated by the above work, a novel RPCA method based on the manifold constraint and non-convex joint sparse learning is proposed to improve the performance of image denoising and background subtraction in this paper. Since manifold constrained learning and joint sparse learning are employed, it is natural to abbreviate it as MCJSL, which to our knowledge has not been systematically analyzed elsewhere in the literature. As is illustrated in [26], compared with L1 norm, L1/2 norm is a better surrogate to measure the original sparsity. Thus, in this paper, a joint sparsity with L2,1/2 norm is proposed by using L2 norm to constrain intra-group structures and L1/2 norm to constrain inter-group structures. To better illustrate the advantage of manifold constraint and L2,1/2 regularization, Fig. 1 shows the framework of the proposed MCJSL. Here, white blocks indicate zero elements, gray blocks represent nonzero elements. It is observed that manifold learning is not only applied to the low-rank component, but also to the sparse component. In summary, manifold constrained learning preserves local geometric structures while non-convex joint sparse learning captures global geometric structures.

In comparison to previous analysis for RPCA-style methods, the merits of this paper can be distilled to the following three aspects:

  • 1.

    Novel manifold constrained model. Instead of considering typical RPCA, a novel MCJSL method is proposed by introducing the manifold constraint and joint sparse learning via L2,1/2 regularization. Actually, it is a very efficient framework for image denoising and background subtraction.

  • 2.

    Optimization algorithm. An effective technique is developed to solve the proposed MCJSL, which is a non-convex constrained minimization problem, by utilizing the manifold alternating direction method of multipliers (MADMM). The resulting subproblems either can be solved by fast solvers or have closed-form solutions. Moreover, the convergence result is analyzed mathematically, which demonstrates that the proposed MCJSL is both efficient and convergent.

  • 3.

    Different computer vision applications. For image denoising, compared with the up-to-data methods, the proposed MCJSL always obtains better performance with respect to peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For background subtraction, the proposed MCJSL also outperforms the traditional state-of-the-art methods in terms of F-score.

The remainder of this paper is organized as follows. Section 2 reviews RPCA and manifold learning. Section 3 establishes the proposed MCJSL with an efficient optimization algorithm. Section 4 implements MCJSL to complete image denoising task. Section 5 applies MCJSL to background subtraction. Finally, Section 6 concludes this paper.

Section snippets

Preliminaries

The purpose of RPCA is to estimate the potential low-rank matrix from its noisy input. Due to the inherent low-rank structure of data in many practical problems, RPCA has obtained great success in many computer vision applications. Here a brief review of the current improvement is provided for this subject. Firstly, a brief review of the fundamental models of RPCA is presented, then the manifold learning is described.

Manifold constrained joint sparse learning

This section presents a novel method called MCJSL, which integrates the manifold constraints and non-convex joint sparsity into RPCA.

MCJSL for image denoising

In this section, the proposed MCJSL is applied to image denoising, where the images are added the Gaussian noise via different degrees. Meanwhile, some excellent methods such as BM3D [33], LSSC [34], NCSR [35], SAIST [36], WNNM [14] and ManiDec [25] are compared.

For a local patch yj in noisy image Y, we can search for its nonlocal similar patches across a relatively large area around it by methods such as manifold projection. By stacking those nonlocal similar patches into a matrix, denoted by D

MCJSL for background subtraction

The purpose of SVD/PCA is to reduce the dimension of the data set while retaining as much information as possible. However, the sparsity of data will be higher with the increase of dimensions. It is more difficult to explore the same data set in high-dimensional vector space than in the same sparse data set. Especially in the traditional PCA algorithm, the measurement of the error depends on the L2 norm fidelity to reduce the noise interference with sparse property. Therefore, the effect of

Conclusions

In this paper, a manifold constrained joint sparse learning (MCJSL) approach is proposed, in which the manifold learning and non-convex regularization are explicitly introduced to constrain continuous spatiotemporal sequences. Technically, the manifold constraints are embedded in the process of low-rank and sparse learning. An effective algorithm for solving the MCJSL is constructed with convergence analysis. The extensive experimental results on image denoising and background subtraction show

CRediT authorship contribution statement

Jingjing Liu: Methodology, Validation, Writing - review & editing. Xianchao Xiu: Supervision, Writing - review & editing. Xin Jiang: Writing - original draft. Wanquan Liu: Validation, Writing - review & editing. Xiaoyang Zeng: Supervision, Writing - review & editing. Mingyu Wang: Investigation. Hui Chen: Data curation, Validation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China under Grant Nos. (12001019, 62074041), and the Natural Science Foundation of Shanghai under Grant Nos. (19ZR1420800, 20ZR1421300). The authors would like to thank the associate editor and three anonymous reviewers for their numerous insightful comments, which have greatly improved the article.

Dr. Jingjing Liu received her Bachelor degree in Electrical Engineering from Zhengzhou University, P.R. China, in 2002, the MSc and PhD degree in Control Science and Engineering at Shanghai University in 2010. She was a joint PhD in Computation department of Curtin University in Australia from December 2015 to December 2016. Currently, she is a candidate of Postdoctoral at the State Key Laboratory of ASIC and System, Fudan University with interests in pattern recognition, computer vision,

References (50)

  • A. Parekh et al.

    Improved sparse low-rank matrix estimation

    Signal Processing

    (2017)
  • G. Liu et al.

    Robust recovery of subspace structures by low-rank representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2012)
  • D. Gross

    Recovering low-rank matrices from few coefficients in any basis

    IEEE Trans. Inf. Theory

    (2011)
  • E.J. Candès et al.

    Robust principal component analysis?

    J. ACM (JACM)

    (2011)
  • J.-F. Cai et al.

    A singular value thresholding algorithm for matrix completion

    SIAM J. Optim.

    (2010)
  • S. Gu et al.

    Weighted nuclear norm minimization and its applications to low level vision

    Int. J. Comput. Vision

    (2017)
  • Y. Hu et al.

    Fast and accurate matrix completion via truncated nuclear norm regularization

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2012)
  • W. Chen

    Simultaneously sparse and low-rank matrix reconstruction via nonconvex and nonseparable regularization

    IEEE Trans. Signal Process.

    (2018)
  • F. Wen et al.

    Robust PCA using generalized nonconvex regularization

    IEEE Trans. Circuits Syst. Video Technol.

    (2019)
  • S. Wang et al.

    Improved RPCA method via non-convex regularisation for image denoising

    IET Signal Proc.

    (2020)
  • J. Liu, S. Ji, J. Ye, Multi-task feature learning via efficient L2,1-norm minimization,...
  • X. Xiu et al.

    Data-driven process monitoring using structured joint sparse canonical correlation analysis

    IEEE Trans. Circuits Syst. II Express Briefs

    (2020)
  • S.T. Roweis et al.

    Nonlinear dimensionality reduction by locally linear embedding

    Science

    (2000)
  • Y. Liu et al.

    Structured joint sparse principal component analysis for fault detection and isolation

    IEEE Trans. Industr. Inf.

    (2018)
  • J. Liu et al.

    ManiDec: Manifold Constrained Low-Rank and Sparse Decomposition

    IEEE Access

    (2019)
  • Cited by (0)

    Dr. Jingjing Liu received her Bachelor degree in Electrical Engineering from Zhengzhou University, P.R. China, in 2002, the MSc and PhD degree in Control Science and Engineering at Shanghai University in 2010. She was a joint PhD in Computation department of Curtin University in Australia from December 2015 to December 2016. Currently, she is a candidate of Postdoctoral at the State Key Laboratory of ASIC and System, Fudan University with interests in pattern recognition, computer vision, machine learning and control systems.

    Dr. Xianchao Xiu received his Bachelor degree in Applied Mathematics from Hebei North University, P.R. China, in 2011, the MSc and PhD degree in Control Theory and Operation Research from Beijing Jiaotong University in 2014 and 2019 respectively. He is currently a post Doctor in State Key Lab for Turbulence and Complex Systems at Peking University. His current research interests include large-scale sparse optimization, signal processing, machine learning, and fault detection.

    Mr. Xin Jiang obtained his Bachelor degree in Microelectronic from Harbin Institute of Technology, P.R. China, in 2019. He is a candidate of Master at the State Key Laboratory of ASIC and System, Fudan University, Shanghai, China with the interests in machine learning algorithm. His current research direction also includes the design of digital integrated circuits based on intelligent algorithms.

    Prof. Wanquan Liu received the BSc degree in Applied Mathematics from Qufu Normal University, P. R. China, in 1985, the MSc degree in Control Theory and Operation Research from Chinese Academy of Science in 1988, and the PhD degree in Electrical Engineering from Shanghai Jiaotong University, in 1993. He once held the ARC Fellowship, U2000 Fellowship and JSPS Fellowship and attracted research funds from different resources. He is currently a Professor in School of Intelligent Systems Engineering at Sun Yat-sen University. He is the Editor-in-chief for the international journal Mathematical Foundation of Computing and in editorial board for seven international journals. His current research interests include large-scale pattern recognition, signal processing, control engineering, machine learning, and computer vision.

    Prof. Xiaoyang Zeng received his PhD degree from Chinese Academy of Sciences in 2001. He is currently the executive director of the State Key Laboratory of ASIC and System the vice dean of the School of Microelectronics of Fudan University. He also serves as the co-chair of the Circuit & System Division of Chinese Institute of Electronics, the steering committee member of ASP-DAC, the TPC Member of A-SSCC, also the TPC Chair of ASICON 2009/2013. His research fields include high-performance and low-power VLSI architecture design for information security algorithms, digital signal processing algorithms, wireless communication base-band processing, and the mixed-signal circuits design technology.

    Prof. Mingyu Wang received the B.S. degree from the Huazhong University of Science and Technology, Wuhan, China, in 1999, the M.S. degree in electronic engineering from Shanghai Jiaotong University, China, in 2001, and the PhD. degree from the School of Microelectronics, Fudan University, Shanghai, China, in 2011. He currently serves as an associate Professor with the State Key Laboratory of ASIC and System, Fudan University. His research interests include artificial intelligence chip design, mixed signal circuit design, and low power SOC design.

    Dr. Hui Chen received the B.S. degree in measurement and control technology and instrument from Jiangsu University in 2006. She received her M.S.degree in measurement and control technology and instrument from Shanghai University in 2009 and her doctor degree in control theory and control engineering from Shanghai University in 2014. She was a visiting Ph.D. Student in the field of robotic in Irseem ESIGELEC, France and Jacobs University Bremen, Germany from 2010 to 2012. She is currently an associate professor in Shanghai University Electric Power. Her research interests include control engineering, computer vision, image processing, and robot navigation.

    1

    These authors contributed equally to this paper.

    View full text