Low-rank tensor train for tensor robust principal component analysis
Introduction
Principal component analysis (PCA), as a classical data analysis and dimension reduction method, has been widely applied in various applications, such as computer vision [1], [2], [3], [4], diffusion magnetic resonance imaging (MRI) [5], [6], hyperspectral image recovery [7], [8], and video recovery [9], [10], [11], [12], [13]. PCA focuses on reconstructing the low-rank component from the original data with noise corruption. According to the dimensions of the data, there are mainly two kinds of PCA methods: the matrix-based method and the tensor-based method.
Matrix-based PCA decomposes a matrix into the sum of a low-rank component Z and a noise component S, i.e., . When S is small multidimensional Gaussian noise, traditional PCA [14] seeks the best rank-k estimate of Z by minimizingHowever, traditional PCA cannot effectively handle large Gaussian noise and severe outliers that are common in practical data. Consequently, robust PCA (RPCA) [15] overcomes this shortcoming by modeling S as a sparse component, i.e.,where denotes the nuclear norm of Z, σr(Z) is the rth singular value of Z, denotes the l1-norm of S and sij is the (i, j)th element of S. The minimization problem (2) is motivated by the fact that nuclear norm and l1-norm provide the tightest convex relaxation for the rank of matrix and l0-norm, respectively.
Tensor PCA focuses on dimension reduction and analysis for high-dimensional data. In practice, we often encounter high-dimensional data, such as color images, videos, and medical data. Traditional RPCA methods process the high-dimensional data by transforming it into a matrix [16]. Such an operation seriously destroys the intrinsic tensor structure of high-dimensional data and increases the computational cost of data analysis. Recently, tensor RPCA (TRPCA) was developed based on tensor algebra [17], [18], [19]. TRPCA decomposes an lth-order tensor into the sum of a low-rank tensor and the sparse noise i.e.,
A central issue in TRPCA is the definition of the tensor rank. However, the definition of a tensor rank is not unique compared with the matrix rank. Two classical tensor rank definitions are CANDECOMP/PARAFA (CP) rank and Tucker rank [20]. CP rank [20] is defined as the smallest number of rank-one tensors formed by the vector outer product. However, the minimization of CP rank is NP-hard, and it is hard to establish a solvable relaxation form for it [21]. Tucker rank [20] is defined aswhere is the mode- matricization of . In order to effectively minimize the Tucker rank, Liu et al. [22] proposed the sum of nuclear norms (SNN), as the convex surrogate of Tucker rank, where are positive constants satisfying . Based on this surrogate, Huang et al. [17] proposed the following TRPCA:where is the sum of the absolute values of all entries in . However, Tucker rank cannot appropriately capture the global correlation of a tensor. The reason is that only a single mode represents the matrix row in Z(i) which is an unbalanced matricization scheme (one mode versus the rest) [23]. For instance, when all the modes have the same dimension (), the dimension of Z(i) is . Looking at the matrix, Tucker and its convex relaxation cannot fully capture the correlation between high-dimensional data. Thus, its low-rankness does not make the optimization problem (4) efficient in addressing the rank optimization problem (3).
Recently, based on the tensor-tensor product and tensor singular value decomposition (t-SVD), the tensor tubal rank and its convex surrogate tensor nuclear norm (TNN) are proposed to characterize the informational and structural complexity of multilinear data [24], [25]. For a third-order tensor Lu et al. [18] applied TNN to TRPCAwhere is the ith frontal slice of and fft denotes the Fast Fourier Transform; see more details in [24], [26]. By the definition of TNN, the correlations along the first and the second modes are characterized by the t-SVD while that along the third mode is encoded by the embedded circular convolution [27]. This implies that the TNN lacks a direct measure of the low-rankness of the third dimension.
More recently, the tensor train (TT) rank has become an active research topic thanks to its definition from a well-balanced matricization scheme. For an lth-order tensor the TT rank is defined aswhere is the mode- matricization of (see Section 2.1). It is worth reminding that Z[i] is obtained by matricizing along the first k modes and the rest modes. Compared with Tucker rank, TT rank can complement the correlations between different modes, by providing the mean of the correlation between a few modes (rather than a single mode) and the rest of the tensor. Inspired by its desired nature, Lee and Cichocki [28] used low TT rank for the singular value decomposition (SVD) of large-scale matrices. Rauhut et al. [29] used low TT rank to achieve the steepest descent iteration of the large-scale least-squares problem. Directly minimizing the TT rank is NP-hard. Thus, TT nuclear norm (TTNN) [23], as the convex surrogate of the TT rank, is defined as . Particularly, Bengua et al. [23] applied TTNN to the low-rank tensor completion problem with good performance.
In this paper, we incorporate the advantages of TTNN into the TRPCA problem by considering the following TTNN-based TRPCA model:where αi are positive weight parameters satisfying λ is a positive parameter. The alternating direction method of multipliers (ADMM) algorithm is developed to solve the proposed model. Moreover, a tensor augmentation technique ket augmentation (KA) is introduced to enhance the performance of our method. Numerical experiments are conducted on synthetic data including the recovery of color images, MRI images, hyperspectral images, and color videos. It is worth mentioning that the problems of rain streaks removal of videos and stripe noise removal of hyperspectral images are also tested to prove the effectiveness of the proposed method. Extensive numerical experiments reveal the superiority of the proposed method over the compared methods.
The paper proceeds as follow. In Section 2, we introduce the corresponding notations and preliminaries. In Section 3, we apply the ADMM to solve the proposed model. In Section 4, numerical experiments are reported. Finally, we summarize this paper in Section 5.
Section snippets
Notations and preliminaries
In this section, we describe the notations and preliminaries used throughout the paper.
The proposed algorithm
In this section, we develop ADMM [33], [34], [35] for solving the convex optimization problem (6). First, we covert (6) to the following problem by introducing auxiliary variables and :The linear constraints can be reformulated as the following matrix-vector multiplication form:where I denotes the identify matrix, and denote the vectorization
Numerical experiments
In this section, we evaluate the performance of the proposed TTNN-based method (denoted as “TTNN”) for restoring observed high-dimensional images as simulation experiments including color images, MRI images, hyperspectral images, and color videos. We also test the real world video rain streaks and hyperspectral image stripes removal problems. We compare the results with two TRPCA methods, including the method based on Tucker rank [41] (denoted as “SNN”) and the method based on tensor tubal rank
Conclusion
TRPCA focuses on efficiently recovering low-rank and sparse components from the observed high-dimensional data. The key of TRPCA is to characterize the low-rankness of tensors. In this paper, we introduced TTNN into the TRPCA problem by taking into full consideration the global correlation of the high-dimensional data. The ADMM have been designed to solve the proposed model. In the simulations with various high-dimensional data sets, TTNN showed better results than SNN and TNN, which are two of
Acknowledgments
The research is supported by National Natural Science Foundation of China (61876203, 61772003, 11901450), the Fundamental Research Funds for the Central Universities (31020180QD126), and National Postdoctoral Program for Innovative Talents (BX20180252). We would like to thank Canyi Lu for providing the codes of the compared methods in [18].
References (43)
- et al.
Total variation and high-order total variation adaptive model for restoring blurred images with cauchy noise
Comput. Math. Appl.
(2019) - et al.
Mf-lrtc: multi-filters guided low-rank tensor coding for image restoration research article
Neurocomputing
(2018) - et al.
Tensor completion using total variation and low-rank matrix factorization
Inf. Sci.
(2016) - et al.
A total variation and group sparsity based tensor optimization model for video rain streak removal
Signal Process. Image Commun.
(2019) - et al.
Low-rank tensor completion via smooth matrix factorization
Appl. Math. Model.
(2019) - et al.
A block coordinate descent approach for sparse principal component analysis
Neurocomputing
(2015) - et al.
Tensor completion via a multi-linear low-n-rank factorization model
Neurocomputing
(2014) - et al.
Total variation with overlapping group sparsity for deblurring images under cauchy noise
Appl. Math. Comput.
(2019) On the pointwise iteration-complexity of a dynamic regularized ADMM with over-relaxation stepsize
Appl. Math. Comput.
(2018)- et al.
Group-based image decomposition using 3-D cartoon and texture priors
Inf. Sci.
(2016)
Convergence analysis of the augmented Lagrange multiplier algorithm for a class of matrix compressive recovery
Appl. Math. Lett.
Fractional-order total variation image denoising based on proximity algorithm
Appl. Math. Comput.
Generalized total variation-based MRI Rician denoising model with spatially adaptive regularization parameters
Magn. Reson. Imaging
A nonconvex relaxation approach to low-rank tensor completion
IEEE Trans. Neural Netws. Learn. Syst.
Bayesian CP factorization of incomplete tensors with automatic rank determination
IEEE Trans. Pattern Anal.
Robust volume minimization-based matrix factorization for remote sensing and document clustering
IEEE Trans. Signal Process.
The role of diffusion magnetic resonance imaging in Parkinson’s disease and in the differential diagnosis with atypical parkinsonism
Radiol. Bras.
Remote sensing image stripe noise removal: from image decomposition perspective
IEEE Trans. Geosci. Remote Sens.
Destriping of multispectral remote sensing image using low-rank tensor decomposition
IEEE J. STARS
A novel video rain streak removal method using directional gradient priors
IEEE Trans. Image Process.
Robust quaternion matrix completion with applications to image inpainting
Numer. Linear Algebra
Cited by (99)
Tensor completion via joint reweighted tensor Q-nuclear norm for visual data recovery
2024, Signal ProcessingA fast correction approach to tensor robust principal component analysis
2024, Applied Mathematical ModellingMulti-layer additive tensor decomposition of infrared video for titanium alloy tensile testing
2023, Engineering Applications of Artificial IntelligenceEfficient low-rank regularization-based algorithms combining advanced techniques for solving tensor completion problems with application to color image recovering
2023, Journal of Computational and Applied MathematicsQuaternion tensor singular value decomposition using a flexible transform-based approach
2023, Signal Processing