Robust low tubal rank tensor completion via factor tensor norm minimization

Highlights

• We give the definitions of tensor double norm and tensor Frobenius/nuclear hybrid norm, and regard them as low-rank regularization penalty of tensor completion.

• We prove that tensor double norm and tensor Frobenius/nuclear hybrid norm are equivalent to Schatten-1/2 and 2/3 quasi-norm.

• We transform a nonconvex problem into two convex subproblems and give an efficient algorithm to solve the model.

• Several experiments on both synthesized data and real world data can verify the superiority of the proposed algorithms from both accuracy and time consumption.

Abstract

Recent research has demonstrated that low tubal rank recovery based on tensor has received extensive attention. In this correspondence, we define tensor double nuclear norm and tensor Frobenius/nuclear hybrid norm to induce a surrogate for tensor tubal rank, and prove that they are equivalent to tensor Schatten-$p$ norm for $p=1/2$ and $p=2/3$. Based on the definition, we propose two novel tractable tensor completion models called Double Nuclear norm regularized Tensor Completion (DNTC) and Frobenius/Nuclear hybrid norm regularized Tensor Completion (FNTC) by integrating these two norm minimization and factorization methods into a joint learning framework. Furthermore, we adopt invertible linear transforms to obtain low tubal rank tensors, which makes the model more flexible and effective. Two efficient algorithms are designed to solve the proposed tensor completion models by incorporating the convexity of the factor norms. Comprehensive experiments are conducted on synthetic and real datasets to achieve better results in comparison with some state-of-the-art approaches.

Introduction

As a higher-order matrix generalization, tensors [1], [2] are important data formats for multi-dimensional data applications. Traditional matrix-based data have been extensively studied and applied. However, many data are usually high-order tensor structures. For instance, color images can be regarded as a third-order tensor with row, column, and color channel, and grayscale videos have both spatial and temporal structures, which is a third-order tensor. Tensor data may be missed or severely corrupted partly in many real-life data. This leads us to study the tensor completion problem (TC) [3], [4], which obtains the missing values of tensors from its partial observations. Usually, the tensor completion problem is an inverse problem [5], [6], which is ill posed without prior knowledge. The most widely adopted prior knowledge is low rank. The prior knowledge of a low rank is inherent in multimedia tensor data [7], [8]. This idea has been applied to a variety of vision and learning tasks, including but not limited to recommendation systems [9], image/video inpainting [8], and hyperspectral data recovery[10].

The rank of a matrix is defined as the number of non-zero singular values, which can be easily achieved by singular value decomposition. The nuclear norm is the convex envelope of the matrix rank on the unit ball of the spectral norm. Matrix completion [11] has received a considerable amount of attention in both theoretical and applied areas [12], [13]. However, as a higher-order generalization of matrix completion, the research on tensor completion is less developed compared with that on matrix completion. This can be attributed to the big difference between the definition of the rank of matrices and that of tensors. Many researchers have been defined tensor rank based on the different tensor decompositions, such as the CANDECOMP/PARAFAC (CP) rank based on CP decomposition [14], Tucker rank based on the Tucker Decomposition [15], Tensor Train (TT) rank based on TT Decomposition [16], and tubal rank based on tensor singular value decomposition (t-SVD) [17]. Based on the different the definitions of tensor ranks, various tensor nuclear norms have been proposed to replace the rank. Friedland et al. [18] defined a tensor nuclear norm based on CP (cTNN) as the convex envelope of CP rank [19]. Liu et al.[20] presented the sum of nuclear norms (SNN), a convex envelop of the Tucker rank, to approximate the Tucker rank. Semerci [21] et al. defined a new tensor nuclear norm (TNN) based on Tubal rank. Furthermore, Zhang [22] et al. applied TNN to tensor completion and obtained the most advanced video inpainting results. Lu et al. [10], [23] performed tensor robust principal component analysis with TNN based on t-SVD by using Fourier transform. However, owing to the limitation of Fourier transform, the lower tubal rank tensor may not be obtained by Fourier transform along each tube. Recently, robust tensor completion has been studied by using cosine transform and any unitary matrix instead of Fourier transform for t-product and t-SVD [24], [25].

Despite the perfect mathematical theory, there exist two problems that limit the application of TNN-based methods: 1) TNN as the convex relaxation of tubal rank can lead to a biased solution owing to the shrinking different singular values equally with the same value. 2) Singular value decomposition (SVD) of large-scale tensors leads to high computational cost. Using nonconvex rank relaxation [26] instead of nuclear norm is a popular choice to address the first shortcoming. To solve the second shortcoming, a large-scale tensor is decomposed into two smaller factor tensors [10] by the decomposition method.

Research shows that it is better to use nonconvex ${\ell }_p$-norm, which is defined as ${\parallel \mathbf{x}\parallel }_p=({\sum }_i|x_i{{|}^p)}^{1/p}$, for $\mathbf{x}={[x_1,x_2,\dots ,x_n]}^{\top }$ and $0<p<1$, to approximate ${\ell }_0$-norm (the number of non-zero entries of a vector) [27], [28] and Schatten-$p$ norm, which is defined as the ${\ell }_p$-norm of the singular values ${\left({\sum }_i{\sigma }_i^p\right)}^{1/p}$, to relax rank function [29], [30] than ${\ell }_1$-norm and nuclear norm as the alternatives. It has been proven theoretically that the Schatten-$p$ norm with small $p$ needs much less observation terms than the nuclear norm minimization for the matrix completion question. Unfortunately, solving a model with the Schatten-$p$ norm directly is unsuitable owing to expensive computational complexity at each iteration, which involves SVD with large scale matrices. Excellent image restoration performance has been achieved using matrix decomposition technology [31]. However, a matrix-based model must rearrange three-order data in a matrix form, which will corrupt the information structure of the real color image and video. In this paper, the real color image and video are regarded as third-order tensors, each front slice of which corresponds to a color channel, and the low tubal rank minimization method is introduced to extract spatial information by using the invertible linear transform. Inspired by the unbiasedness and decomposability of the nonconvex Schatten-$p$ norm, we propose a novel color image and video restoration model, called robust low tubal rank tensor completion via factor tensor norm minimization, which extends the nuclear norm to specific Schatten-$p$ norms for $p=1/2$ and $2/3$. Based on the Schatten-$1/2$ and $2/3$ norms of tensor decomposition, we provide the convex substitution of these two norms as tensor double norm and tensor Forbenius/nucleus hybrid norm, respectively. Consequently, computational complexity is reduced by calculating the singular value decomposition of small-scale matrix, and each subproblem has a closed form solution.