research-article

High-Order Tensor Recovery Coupling Multilayer Subspace Priori with Application in Video Restoration

Authors:

Jianjun WangAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 1212 - 1220

https://doi.org/10.1145/3581783.3613790

Published: 27 October 2023 Publication History

Abstract

In the real world, a large amount of high-order tensor data (order>3) exists, such as color videos, multispectral videos, and light-field images. However, these data often face challenges in transportation, storage, and susceptibility to damage. Meanwhile, most existing tensor-based information processing methods only concentrate on third-order tensors, which may not meet the complex requirements of high-dimensional data processing. In this paper, to better address the high-order tensor recovery issue, we propose a novel method that couples multilayer subspace priors with high-order tensor recovery techniques for tensor completion and robust tensor principal component analysis. Moreover, we provide theoretical guarantees for our approach's recovery and demonstrate that it achieves comparable performance under weaker incoherent conditions. Additionally, we develop two efficient and interpretable algorithms based on the alternating direction method of multipliers (ADMM) to solve our model. Owing to the adaptability of subspace prior information, our method demonstrates superior performance in recovering various types of data, including color videos and multispectral videos, compared with various advanced algorithms currently available.

References

[1]

Johann A Bengua, Ho N Phien, Hoang Duong Tuan, and Minh N Do. 2017. Efficient tensor completion for color image and video recovery: Low-rank tensor train. IEEE Transactions on Image Processing, Vol. 26, 5 (2017), 2466--2479.

Digital Library

[2]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning, Vol. 3, 1 (2011), 1--122.

[3]

Wenfei Cao, Yao Wang, Jian Sun, Deyu Meng, Can Yang, Andrzej Cichocki, and Zongben Xu. 2016. Total variation regularized tensor RPCA for background subtraction from compressive measurements. IEEE Transactions on Image Processing, Vol. 25, 9 (2016), 4075--4090.

Digital Library

[4]

Maolin Che, Andrzej Cichocki, and Yimin Wei. 2017. Neural networks for computing best rank-one approximations of tensors and its applications. Neurocomputing, Vol. 267 (2017), 114--133.

[5]

Kai-Yang Chiang, Cho-Jui Hsieh, and Inderjit Dhillon. 2016. Robust principal component analysis with side information. In International Conference on Machine Learning. PMLR, 2291--2299.

[6]

Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on image processing, Vol. 16, 8 (2007), 2080--2095.

[7]

Armin Eftekhari, Dehui Yang, and Michael B Wakin. 2018. Weighted matrix completion and recovery with prior subspace information. IEEE Transactions on Information Theory, Vol. 64, 6 (2018), 4044--4071.

[8]

Jicong Fan. 2022. Multi-Mode Deep Matrix and Tensor Factorization. In International Conference on Learning Representations. https://openreview.net/forum?id=6YVIk0sAkF_

[9]

Athinodoros S. Georghiades, Peter N. Belhumeur, and David J. Kriegman. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE transactions on pattern analysis and machine intelligence, Vol. 23, 6 (2001), 643--660.

[10]

Christopher J Hillar and Lek-Heng Lim. 2013. Most tensor problems are NP-hard. Journal of the ACM (JACM), Vol. 60, 6 (2013), 1--39.

Digital Library

[11]

Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, and Liang-Jian Deng. 2020. Multi-dimensional imaging data recovery via minimizing the partial sum of tubal nuclear norm. J. Comput. Appl. Math., Vol. 372 (2020), 112680.

Digital Library

[12]

Misha E Kilmer, Karen Braman, Ning Hao, and Randy C Hoover. 2013. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl., Vol. 34, 1 (2013), 148--172.

Digital Library

[13]

Misha E Kilmer and Carla D Martin. 2011. Factorization strategies for third-order tensors. Linear Algebra Appl., Vol. 435, 3 (2011), 641--658.

[14]

Tamara G Kolda and Brett W Bader. 2009. Tensor decompositions and applications. SIAM review, Vol. 51, 3 (2009), 455--500.

[15]

Weichao Kong, Feng Zhang, Wenjin Qin, and Jianjun Wang. 2023. Low-Tubal-Rank Tensor Recovery with Multilayer Subspace Prior Learning. Pattern Recognition (2023), 109545.

[16]

Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye. 2012. Tensor completion for estimating missing values in visual data. IEEE transactions on pattern analysis and machine intelligence, Vol. 35, 1 (2012), 208--220.

[17]

Zhen Long, Yipeng Liu, Longxi Chen, and Ce Zhu. 2019. Low rank tensor completion for multiway visual data. Signal processing, Vol. 155 (2019), 301--316.

[18]

Zhen Long, Ce Zhu, Jiani Liu, and Yipeng Liu. 2021. Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, Vol. 30 (2021), 3568--3580.

Digital Library

[19]

Canyi Lu. 2021. Transforms based tensor robust pca: Corrupted low-rank tensors recovery via convex optimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1145--1152.

[20]

Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. 2019a. Tensor robust principal component analysis with a new tensor nuclear norm. IEEE transactions on pattern analysis and machine intelligence, Vol. 42, 4 (2019), 925--938.

[21]

Canyi Lu, Jiashi Feng, Zhouchen Lin, and Shuicheng Yan. 2018. Exact low tubal rank tensor recovery from Gaussian measurements. arXiv preprint arXiv:1806.02511 (2018).

[22]

Canyi Lu, Peng Xi, and Wei Yunchao. 2019b. Low-Rank Tensor Completion With a New Tensor Nuclear Norm Induced by Invertible Linear Transforms. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR).

[23]

Haiping Lu, Konstantinos N Plataniotis, and Anastasios N Venetsanopoulos. 2006. Multilinear principal component analysis of tensor objects for recognition. In 18th International Conference on Pattern Recognition (ICPR'06), Vol. 2. IEEE, 776--779.

[24]

Carla D Martin, Richard Shafer, and Betsy LaRue. 2013. An order-p tensor factorization with applications in imaging. SIAM Journal on Scientific Computing, Vol. 35, 1 (2013), A474--A490.

Digital Library

[25]

Michael K Ng, Xiongjun Zhang, and Xi-Le Zhao. 2020. Patched-tube unitary transform for robust tensor completion. Pattern Recognition, Vol. 100 (2020), 107181.

Digital Library

[26]

Wenjin Qin, Hailin Wang, Feng Zhang, Mingwei Dai, and Jianjun Wang. 2021. Robust low-rank tensor reconstruction using high-order t-SVD. Journal of Electronic Imaging, Vol. 30, 6 (2021), 063016-063016.

[27]

Wenjin Qin, Hailin Wang, Feng Zhang, Jianjun Wang, Xin Luo, and Tingwen Huang. 2022. Low-rank high-order tensor completion with applications in visual data. IEEE Transactions on Image Processing, Vol. 31 (2022), 2433--2448.

[28]

Yaniv Romano, Michael Elad, and Peyman Milanfar. 2017. The little engine that could: Regularization by denoising (RED). SIAM Journal on Imaging Sciences, Vol. 10, 4 (2017), 1804--1844.

Digital Library

[29]

Leonid I Rudin, Stanley Osher, and Emad Fatemi. 1992. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, Vol. 60, 1--4 (1992), 259--268.

[30]

Hailin Wang, Jiangjun Peng, Wenjin Qin, Jianjun Wang, and Deyu Meng. 2023. Guaranteed Tensor Recovery Fused Low-rankness and Smoothness. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

Digital Library

[31]

Xuezhong Wang, Maolin Che, and Yimin Wei. 2019. Neural networks based approach solving multi-linear systems with M-tensors. Neurocomputing, Vol. 351 (2019), 33--42.

Digital Library

[32]

Yao Wang, Deyu Meng, and Ming Yuan. 2018. Sparse recovery: from vectors to tensors. National Science Review, Vol. 5, 5 (2018), 756--767.

[33]

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, Vol. 13, 4 (2004), 600--612.

Digital Library

[34]

Qi Xie, Qian Zhao, Deyu Meng, Zongben Xu, Shuhang Gu, Wangmeng Zuo, and Lei Zhang. 2016. Multispectral images denoising by intrinsic tensor sparsity regularization. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1692--1700.

[35]

Honghui Xu, Jianwei Zheng, Xiaomin Yao, Yuchao Feng, and Shengyong Chen. 2021. Fast tensor nuclear norm for structured low-rank visual inpainting. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 2 (2021), 538--552.

[36]

Jing-Hua Yang, Xi-Le Zhao, Tian-Hui Ma, Yong Chen, Ting-Zhu Huang, and Meng Ding. 2020. Remote sensing images destriping using unidirectional hybrid total variation and nonconvex low-rank regularization. J. Comput. Appl. Math., Vol. 363 (2020), 124--144.

Digital Library

[37]

Jinchun Zhan and Namrata Vaswani. 2015. Robust PCA with partial subspace knowledge. IEEE Transactions on Signal Processing, Vol. 63, 13 (2015), 3332--3347.

Digital Library

[38]

Feng Zhang, Jianjun Wang, Wendong Wang, and Chen Xu. 2020. Low-tubal-rank plus sparse tensor recovery with prior subspace information. IEEE transactions on pattern analysis and machine intelligence, Vol. 43, 10 (2020), 3492--3507.

[39]

Xiongjun Zhang and Michael K Ng. 2019. A corrected tensor nuclear norm minimization method for noisy low-rank tensor completion. SIAM Journal on Imaging Sciences, Vol. 12, 2 (2019), 1231--1273.

Digital Library

[40]

Zemin Zhang and Shuchin Aeron. 2016. Exact tensor completion using t-SVD. IEEE Transactions on Signal Processing, Vol. 65, 6 (2016), 1511--1526.

Digital Library

[41]

Zemin Zhang, Gregory Ely, Shuchin Aeron, Ning Hao, and Misha Kilmer. 2014. Novel methods for multilinear data completion and de-noising based on tensor-SVD. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3842--3849.

Digital Library

[42]

Yu-Bang Zheng, Ting-Zhu Huang, Xi-Le Zhao, Tai-Xiang Jiang, Tian-Hui Ma, and Teng-Yu Ji. 2019. Mixed noise removal in hyperspectral image via low-fibered-rank regularization. IEEE Transactions on Geoscience and Remote Sensing, Vol. 58, 1 (2019), 734--749.

Cited By

He CXu YWu ZZheng SWei Z(2024)Multi-Dimensional Visual Data Restoration: Uncovering the Global Discrepancy in Transformed High-Order Tensor Singular ValuesIEEE Transactions on Image Processing10.1109/TIP.2024.347573833(6409-6424)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3475738

Index Terms

High-Order Tensor Recovery Coupling Multilayer Subspace Priori with Application in Video Restoration
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction

Recommendations

Low-Tubal-Rank tensor recovery with multilayer subspace prior learning
Highlights
- Under the t-SVD framework, by means of the nonuniqueness of t-SVD and the few large singular values contain the main tensor information, we first integrate ...
Abstract
Currently, low-rank tensor recovery employing the subspace prior information is an emerging topic, which has attracted considerable attention. However, existing studies cannot flexibly and fully utilize the accessible subspace prior ...
Iterative Adaptive Nonconvex Low-Rank Tensor Approximation to Image Restoration Based on ADMM

In this paper, in order to recover more finer details of the image and to avoid the loss of image structure information for image restoration problem, we develop an iterative adaptive weighted core tensor thresholding (IAWCTT) approach based on the ...
Tensor recovery from binary measurements fused low-rankness and smoothness
Abstract
Tensor compressed sensing (TCS) from binary measurements aims to recover a tensor with low-rankness from the binary quantization of its degraded linear measurements, which inherits the advantages of low equipment cost and high sampling efficiency ...
Highlights
- A method for tensor recovery from binary measurements which codes both low-rankness and smoothness was proposed.
- We prove a recovery guarantee for the method and provide an explicit selection rule for the regularization parameter.
- ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Chongqing
Graduate Student Scientific Research Innovation Projects in Southwest University
Fundamental Research Funds for the Central Universities
National Natural Science Foundation of China
China Postdoctoral Science Foundation

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
211
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)19

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

He CXu YWu ZZheng SWei Z(2024)Multi-Dimensional Visual Data Restoration: Uncovering the Global Discrepancy in Transformed High-Order Tensor Singular ValuesIEEE Transactions on Image Processing10.1109/TIP.2024.347573833(6409-6424)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3475738

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten