Low-rank tensor train for tensor robust principal component analysis

https://doi.org/10.1016/j.amc.2019.124783Get rights and content

Abstract

Recently, tensor train rank, defined by a well-balanced matricization scheme, has been shown the powerful capacity to capture the hidden correlations among different modes of a tensor, leading to great success in tensor completion problem. Most of the high-dimensional data in the real world are more likely to be grossly corrupted with sparse noise. In this paper, based on tensor train rank, we consider a new model for tensor robust principal component analysis which aims to recover a low-rank tensor corrupted by sparse noise. The alternating direction method of multipliers algorithm is developed to solve the proposed model. A tensor augmentation tool called ket augmentation is used to convert lower-order tensors to higher-order tensors to enhance the performance of our method. Experiments of simulated data show the superiority of the proposed method in terms of PSNR and SSIM values. Moreover, experiments of the real rain streaks removal and the real stripe noise removal also illustrate the effectiveness of the proposed method.

Introduction

Principal component analysis (PCA), as a classical data analysis and dimension reduction method, has been widely applied in various applications, such as computer vision [1], [2], [3], [4], diffusion magnetic resonance imaging (MRI) [5], [6], hyperspectral image recovery [7], [8], and video recovery [9], [10], [11], [12], [13]. PCA focuses on reconstructing the low-rank component from the original data with noise corruption. According to the dimensions of the data, there are mainly two kinds of PCA methods: the matrix-based method and the tensor-based method.

Matrix-based PCA decomposes a matrix DRn1×n2 into the sum of a low-rank component Z and a noise component S, i.e., D=Z+S. When S is small multidimensional Gaussian noise, traditional PCA [14] seeks the best rank-k estimate of Z by minimizingargminZDZF2,s.t.rank(Z)k.However, traditional PCA cannot effectively handle large Gaussian noise and severe outliers that are common in practical data. Consequently, robust PCA (RPCA) [15] overcomes this shortcoming by modeling S as a sparse component, i.e.,argminZ,SZ*+S1,s.t.D=Z+S,where Z*=rσr(Z) denotes the nuclear norm of Z, σr(Z) (r=1,2,,min(n1,n2)) is the rth singular value of Z, S1=ij|sij| denotes the l1-norm of S and sij is the (i, j)th element of S. The minimization problem (2) is motivated by the fact that nuclear norm and l1-norm provide the tightest convex relaxation for the rank of matrix and l0-norm, respectively.

Tensor PCA focuses on dimension reduction and analysis for high-dimensional data. In practice, we often encounter high-dimensional data, such as color images, videos, and medical data. Traditional RPCA methods process the high-dimensional data by transforming it into a matrix [16]. Such an operation seriously destroys the intrinsic tensor structure of high-dimensional data and increases the computational cost of data analysis. Recently, tensor RPCA (TRPCA) was developed based on tensor algebra [17], [18], [19]. TRPCA decomposes an lth-order tensor DRn1×n2××nl into the sum of a low-rank tensor Z and the sparse noise S, i.e.,argminZ,Srank(Z)+λS0,s.t.D=Z+S.

A central issue in TRPCA is the definition of the tensor rank. However, the definition of a tensor rank is not unique compared with the matrix rank. Two classical tensor rank definitions are CANDECOMP/PARAFA (CP) rank and Tucker rank [20]. CP rank [20] is defined as the smallest number of rank-one tensors formed by the vector outer product. However, the minimization of CP rank is NP-hard, and it is hard to establish a solvable relaxation form for it [21]. Tucker rank [20] is defined asranktc(Z):=(rank(Z(1)),rank(Z(2)),,rank(Z(l))),where Z(i)Rni×(n1ni1ni+1nl) is the mode-i matricization of Z. In order to effectively minimize the Tucker rank, Liu et al. [22] proposed the sum of nuclear norms (SNN), i=1lαiZ(i)*, as the convex surrogate of Tucker rank, where {αi}i=1l are positive constants satisfying i=1lαi=1. Based on this surrogate, Huang et al. [17] proposed the following TRPCA:argminZ,Si=1lαiZ(i)*+λS1,s.t.D=Z+S,where S1 is the sum of the absolute values of all entries in S. However, Tucker rank cannot appropriately capture the global correlation of a tensor. The reason is that only a single mode represents the matrix row in Z(i) which is an unbalanced matricization scheme (one mode versus the rest) [23]. For instance, when all the modes have the same dimension (n1==nl=n), the dimension of Z(i) is n×nl1. Looking at the matrix, Tucker and its convex relaxation cannot fully capture the correlation between high-dimensional data. Thus, its low-rankness does not make the optimization problem (4) efficient in addressing the rank optimization problem (3).

Recently, based on the tensor-tensor product and tensor singular value decomposition (t-SVD), the tensor tubal rank and its convex surrogate tensor nuclear norm (TNN) are proposed to characterize the informational and structural complexity of multilinear data [24], [25]. For a third-order tensor ZRn1×n2×n3, Lu et al. [18] applied TNN to TRPCAargminZ,Si=1n3αiZ¯(i)*+λS1,s.t.D=Z+S,where Z¯(i) is the ith frontal slice of Z¯=fft(Z,[],3) and fft denotes the Fast Fourier Transform; see more details in [24], [26]. By the definition of TNN, the correlations along the first and the second modes are characterized by the t-SVD while that along the third mode is encoded by the embedded circular convolution [27]. This implies that the TNN lacks a direct measure of the low-rankness of the third dimension.

More recently, the tensor train (TT) rank has become an active research topic thanks to its definition from a well-balanced matricization scheme. For an lth-order tensor ZRn1×n2××nl, the TT rank is defined asranktt(Z):=(rank(Z[1]),rank(Z[2]),,rank(Z[l1])),where Z[i]RΠk=1ink×Πk=i+1lnk is the mode-(1,2,,k) matricization of Z (see Section 2.1). It is worth reminding that Z[i] is obtained by matricizing along the first k modes and the rest lk modes. Compared with Tucker rank, TT rank can complement the correlations between different modes, by providing the mean of the correlation between a few modes (rather than a single mode) and the rest of the tensor. Inspired by its desired nature, Lee and Cichocki [28] used low TT rank for the singular value decomposition (SVD) of large-scale matrices. Rauhut et al. [29] used low TT rank to achieve the steepest descent iteration of the large-scale least-squares problem. Directly minimizing the TT rank is NP-hard. Thus, TT nuclear norm (TTNN) [23], as the convex surrogate of the TT rank, is defined as Z*=i=1l1αiZ[i]*. Particularly, Bengua et al. [23] applied TTNN to the low-rank tensor completion problem with good performance.

In this paper, we incorporate the advantages of TTNN into the TRPCA problem by considering the following TTNN-based TRPCA model:argminZ,Si=1l1αiZ[i]*+λS1,s.t.D=Z+S,where αi are positive weight parameters satisfying i=1l1αi=1, λ is a positive parameter. The alternating direction method of multipliers (ADMM) algorithm is developed to solve the proposed model. Moreover, a tensor augmentation technique ket augmentation (KA) is introduced to enhance the performance of our method. Numerical experiments are conducted on synthetic data including the recovery of color images, MRI images, hyperspectral images, and color videos. It is worth mentioning that the problems of rain streaks removal of videos and stripe noise removal of hyperspectral images are also tested to prove the effectiveness of the proposed method. Extensive numerical experiments reveal the superiority of the proposed method over the compared methods.

The paper proceeds as follow. In Section 2, we introduce the corresponding notations and preliminaries. In Section 3, we apply the ADMM to solve the proposed model. In Section 4, numerical experiments are reported. Finally, we summarize this paper in Section 5.

Section snippets

Notations and preliminaries

In this section, we describe the notations and preliminaries used throughout the paper.

The proposed algorithm

In this section, we develop ADMM [33], [34], [35] for solving the convex optimization problem (6). First, we covert (6) to the following problem by introducing auxiliary variables Ui(i=1,2,,l1) and Y:argminZ,Si=1l1αiUi*+λY1,s.t.Ui=Z[i],D=Z+S,Y=S.The linear constraints can be reformulated as the following matrix-vector multiplication form:(II0II0I0)(zs)+(0000I0000I00000I)(yu1u2ul1)=(d000),where I denotes the identify matrix, z, s, y, {ui}i=1l1 and d denote the vectorization

Numerical experiments

In this section, we evaluate the performance of the proposed TTNN-based method (denoted as “TTNN”) for restoring observed high-dimensional images as simulation experiments including color images, MRI images, hyperspectral images, and color videos. We also test the real world video rain streaks and hyperspectral image stripes removal problems. We compare the results with two TRPCA methods, including the method based on Tucker rank [41] (denoted as “SNN”) and the method based on tensor tubal rank

Conclusion

TRPCA focuses on efficiently recovering low-rank and sparse components from the observed high-dimensional data. The key of TRPCA is to characterize the low-rankness of tensors. In this paper, we introduced TTNN into the TRPCA problem by taking into full consideration the global correlation of the high-dimensional data. The ADMM have been designed to solve the proposed model. In the simulations with various high-dimensional data sets, TTNN showed better results than SNN and TNN, which are two of

Acknowledgments

The research is supported by National Natural Science Foundation of China (61876203, 61772003, 11901450), the Fundamental Research Funds for the Central Universities (31020180QD126), and National Postdoctoral Program for Innovative Talents (BX20180252). We would like to thank Canyi Lu for providing the codes of the compared methods in [18].

References (43)

  • C. Li et al.

    Convergence analysis of the augmented Lagrange multiplier algorithm for a class of matrix compressive recovery

    Appl. Math. Lett.

    (2016)
  • D.L. Chen et al.

    Fractional-order total variation image denoising based on proximity algorithm

    Appl. Math. Comput.

    (2015)
  • R.W. Liu et al.

    Generalized total variation-based MRI Rician denoising model with spatially adaptive regularization parameters

    Magn. Reson. Imaging

    (2014)
  • X.J. Zhang

    A nonconvex relaxation approach to low-rank tensor completion

    IEEE Trans. Neural Netws. Learn. Syst.

    (2018)
  • Q.B. Zhao et al.

    Bayesian CP factorization of incomplete tensors with automatic rank determination

    IEEE Trans. Pattern Anal.

    (2015)
  • X. Fu et al.

    Robust volume minimization-based matrix factorization for remote sensing and document clustering

    IEEE Trans. Signal Process.

    (2016)
  • R.V. Oliveira et al.

    The role of diffusion magnetic resonance imaging in Parkinson’s disease and in the differential diagnosis with atypical parkinsonism

    Radiol. Bras.

    (2017)
  • Y. Chang et al.

    Remote sensing image stripe noise removal: from image decomposition perspective

    IEEE Trans. Geosci. Remote Sens.

    (2016)
  • Y. Chen et al.

    Destriping of multispectral remote sensing image using low-rank tensor decomposition

    IEEE J. STARS

    (2018)
  • T.X. Jiang et al.

    A novel video rain streak removal method using directional gradient priors

    IEEE Trans. Image Process.

    (2019)
  • Z. Jia et al.

    Robust quaternion matrix completion with applications to image inpainting

    Numer. Linear Algebra

    (2019)
  • Cited by (99)

    View all citing articles on Scopus
    View full text