skip to main content
10.1145/3394171.3413911acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Online Multi-view Subspace Learning with Mixed Noise

Published: 12 October 2020 Publication History

Abstract

Multi-view learning reveals the latent correlation between different input modalities and has achieved outstanding performances in many fields. Recent approaches aim to find a low-dimensional subspace to reconstruct each view, in which the gross residual or noise follows either Gaussian or Laplacian distribution. However, the noise distribution is often more complex in practical applications, and a deterministic distribution assumption is incapable of modeling it. Additionally, referring to time-changed data, e.g., videos, the noise is temporal smooth, preventing us from processing the data with the whole input, as have generally been done in many existing multi-view learning methods. To tackle these problems, a novel online multi-view subspace learning is proposed in this paper. Particularly, our proposed method not only estimates a transformation for each view to extract the correlation among various views, but also introduces a Mixture of Gausssians (MoG) model into the multi-view data, successfully exploiting numbers of Gaussian Distributions to adaptively fit a wider range of the complex noise. Furthermore, we further design a novel online Expectation Maximization (EM) algorithm, being capable of efficiently processing the dynamic data. Experimental results substantiate the effectiveness and superiority of our approach.

Supplementary Material

RAR File (mmfp0151aux.rar)
This is the supplementary material for "Online Multi-view Subspace Learning with Mixed Noise".
MP4 File (3394171.3413911.mp4)
Existing multi-view learning subspace methods aim to find a low-dimensional subspace to reconstruct each view, in which the gross residual or noise follows either Gaussian or Laplacian distribution. However, the noise distribution is often more complex in practical applications, and a deterministic distribution assumption is incapable of modeling it. Also they are batch-based methods, resulting in a high computational complexity and a large memory consumption. To tackle these problems, a novel online multi-view subspace learning is proposed, which introduces a Mixture of Gausssians (MoG) model into the multi-view data, successfully exploiting numbers of Gaussian Distributions to adaptively fit a wider range of the complex noise. Furthermore, we further design a novel online Expectation Maximization (EM) algorithm, being capable of efficiently processing the data with only one sample in each epoch. We apply our proposed method to synthetic and real-world data, showing its effectiveness.

References

[1]
J. Berclaz, F. Fleuret, E. Turetken, and P. Fua. 2011. Multiple Object Tracking using K-Shortest Paths Optimization . IEEE Transactions on Pattern Analysis and Machine Intelligence (2011).
[2]
Jiashi Feng, Huan Xu, and Shuicheng Yan. 2013. Online robust pca via stochastic optimization. In Advances in Neural Information Processing Systems. 404--412.
[3]
F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua. 2008. Multi-Camera People Tracking with a Probabilistic Occupancy Map . IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, 2 (February 2008), 267--282.
[4]
Stephen Gillen, Christopher Jung, Michael Kearns, and Aaron Roth. 2018. Online learning with an unknown fairness metric. In Advances in Neural Information Processing Systems. 2600--2609.
[5]
Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, and Simon Baker. 2010. Multi-pie. Image and Vision Computing, Vol. 28, 5 (2010), 807--813.
[6]
Le Han, Zhen Wu, Kui Zeng, and Xiaowei Yang. 2018. Online multilinear principal component analysis. Neurocomputing, Vol. 275 (2018), 888--896.
[7]
Harold Hotelling. 1936. Relations between two sets of variates. Biometrika, Vol. 28, 3/4 (1936), 321--377.
[8]
Jinxing Li, Guangming Lu, Bob Zhang, Jane You, and David Zhang. 2019 a. Shared Linear Encoder-Based Multikernel Gaussian Process Latent Variable Model for Visual Classification. IEEE transactions on cybernetics (2019).
[9]
Jinxing Li, Hongwei Yong, Bob Zhang, Mu Li, Lei Zhang, and David Zhang. 2018a. A probabilistic hierarchical model for multi-view and multi-feature classification. In Thirty-Second AAAI Conference on Artificial Intelligence .
[10]
Jinxing Li, Bob Zhang, Guangming Lu, Hu Ren, and David Zhang. 2018c. Visual classification with multikernel shared Gaussian process latent variable model. IEEE transactions on cybernetics, Vol. 49, 8 (2018), 2886--2899.
[11]
Jinxing Li, Bob Zhang, Guangming Lu, Jane You, Yong Xu, Feng Wu, and David Zhang. 2019 e. Relaxed Asymmetric Deep Hashing Learning: Point-to-Angle Matching. IEEE Transactions on Neural Networks and Learning Systems (2019).
[12]
Jinxing Li, Bob Zhang, Guangming Lu, and David Zhang. 2018b. Shared linear encoder-based Gaussian process latent variable model for visual classification. In Proceedings of the 26th ACM international conference on Multimedia. ACM, 26--34.
[13]
Jinxing Li, Bob Zhang, Guangming Lu, and David Zhang. 2019 d. Generative multi-view and multi-feature learning for classification. Information Fusion, Vol. 45 (2019), 215--226.
[14]
Jinxing Li, Bob Zhang, and David Zhang. 2017a. Shared autoencoder Gaussian process latent variable model for visual classification. IEEE transactions on neural networks and learning systems, Vol. 29, 9 (2017), 4272--4286.
[15]
Jinxing Li, David Zhang, Yongcheng Li, Jian Wu, and Bob Zhang. 2017b. Joint similar and specific learning for diabetes mellitus and impaired glucose regulation detection. Information Sciences, Vol. 384 (2017), 191--204.
[16]
Zhenglai Li, Chang Tang, Jiajia Chen, Cheng Wan, Weiqing Yan, and Xinwang Liu. 2019 b. Diversity and consistency learning guided spectral embedding for multi-view clustering. Neurocomputing, Vol. 370 (2019), 128--139.
[17]
Zhaoqun Li, Cheng Xu, and Biao Leng. 2019 c. Angular triplet-center loss for multi-view 3d shape retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 8682--8689.
[18]
Weifeng Liu, Xinghao Yang, Dapeng Tao, Jun Cheng, and Yuanyan Tang. 2018. Multiview dimension reduction via Hessian multiset canonical correlations. Information Fusion, Vol. 41 (2018), 119--128.
[19]
Xinwang Liu, Miaomiao Li, Chang Tang, Jingyuan Xia, Jian Xiong, Li Liu, Marius Kloft, and En Zhu. 2020. Efficient and Effective Regularized Incomplete Multi-view Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).
[20]
Eric F Lock, Katherine A Hoadley, James Stephen Marron, and Andrew B Nobel. 2013. Joint and individual variation explained (JIVE) for integrated analysis of multiple data types. The annals of applied statistics, Vol. 7, 1 (2013), 523.
[21]
Cewu Lu, Jiaping Shi, and Jiaya Jia. 2013. Online robust dictionary learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 415--422.
[22]
Jing Lu, Doyen Sahoo, Peilin Zhao, and Steven CH Hoi. 2018. Sparse passive-aggressive learning for bounded online kernel methods. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 9, 4 (2018), 45.
[23]
Vladimir Maz'ya and Gunther Schmidt. 1996. On approximate approximations using Gaussian kernels. IMA J. Numer. Anal., Vol. 16, 1 (1996), 13--29.
[24]
Deyu Meng and Fernando Torre. 2013. Robust matrix factorization with unknown noise. In Proceedings of the IEEE International Conference on Computer Vision. 1337--1344.
[25]
Ivan Mizera and Christine H Müller. 2002. Breakdown points of cauchy regression-scale estimators. Statistics & probability letters, Vol. 57, 1 (2002), 79--89.
[26]
Xiaorui Song and Lingda Wu. 2019. A Novel Hyperspectral Endmember Extraction Algorithm Based on Online Robust Dictionary Learning. Remote Sensing, Vol. 11, 15 (2019), 1792.
[27]
Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, Wangmeng Zuo, and Chia-Wen Lin. 2019. Deep learning on image denoising: An overview. arXiv preprint arXiv:1912.13171 (2019).
[28]
Chunwei Tian, Yong Xu, Zuoyong Li, Wangmeng Zuo, Lunke Fei, and Hong Liu. 2020 a. Attention-guided CNN for image denoising. Neural Networks, Vol. 124 (2020), 117--129.
[29]
Chunwei Tian, Yong Xu, Wangmeng Zuo, Bob Zhang, Lunke Fei, and Chia-Wen Lin. 2020 b. Coarse-to-fine cnn for image super-resolution. IEEE Transactions on Multimedia (2020).
[30]
Viivi Uurtio, Sahely Bhadra, and Juho Rousu. 2019. Large-Scale Sparse Kernel Canonical Correlation Analysis. In International Conference on Machine Learning. 6383--6391.
[31]
Jie Wen, Yong Xu, and Hong Liu. 2018a. Incomplete multiview spectral clustering with adaptive graph learning. IEEE transactions on cybernetics (2018).
[32]
Jie Wen, Zheng Zhang, Yong Xu, Bob Zhang, Lunke Fei, and Hong Liu. 2019. Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5393--5400.
[33]
Jie Wen, Zheng Zhang, Zhao Zhang, Lunke Fei, and Meng Wang. 2020. Generalized Incomplete Multiview Clustering With Flexible Locality Structure Diffusion. IEEE Transactions on Cybernetics (2020).
[34]
Jie Wen, Zuofeng Zhong, Zheng Zhang, Lunke Fei, Zhihui Lai, and Runze Chen. 2018b. Adaptive locality preserving regression. IEEE Transactions on Circuits and Systems for Video Technology (2018).
[35]
Wei Xiao, Xiaolin Huang, Fan He, Jorge Silva, Saba Emrani, and Arin Chaudhuri. 2019. Online robust principal component analysis with change point detection. IEEE Transactions on Multimedia, Vol. 22, 1 (2019), 59--68.
[36]
Chang Xu, Dacheng Tao, and Chao Xu. 2015. Multi-view intact space learning. Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 37, 12 (2015), 2531--2544.
[37]
Hongwei Yong, Deyu Meng, Jinxing Li, Wangmeng Zuo, and Lei Zhang. 2018. Model Inconsistent but Correlated Noise: Multi-view Subspace Learning with Regularized Mixture of Gaussians. arXiv preprint arXiv:1811.02934 (2018).
[38]
Changqing Zhang, Huazhu Fu, Qinghua Hu, Xiaochun Cao, Yuan Xie, Dacheng Tao, and Dong Xu. 2018. Generalized latent multi-view subspace clustering. IEEE transactions on pattern analysis and machine intelligence (2018).
[39]
Guoxu Zhou, Andrzej Cichocki, Yu Zhang, and Danilo P Mandic. 2015. Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction. (2015).

Cited By

View all
  • (2024)Refining Graph Structure for Incomplete Multi-View ClusteringIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.318976335:2(2300-2313)Online publication date: Feb-2024
  • (2024)UVaT: Uncertainty Incorporated View-Aware Transformer for Robust Multi-View ClassificationIEEE Transactions on Image Processing10.1109/TIP.2024.345193133(5129-5143)Online publication date: 2024
  • (2023)Debiased Video-Text Retrieval via Soft Positive Sample CalibrationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.324887333:9(5257-5270)Online publication date: Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '20: Proceedings of the 28th ACM International Conference on Multimedia
October 2020
4889 pages
ISBN:9781450379885
DOI:10.1145/3394171
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. expectation maximization
  2. mixture of gausssians
  3. multi-view
  4. online

Qualifiers

  • Research-article

Funding Sources

Conference

MM '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Refining Graph Structure for Incomplete Multi-View ClusteringIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.318976335:2(2300-2313)Online publication date: Feb-2024
  • (2024)UVaT: Uncertainty Incorporated View-Aware Transformer for Robust Multi-View ClassificationIEEE Transactions on Image Processing10.1109/TIP.2024.345193133(5129-5143)Online publication date: 2024
  • (2023)Debiased Video-Text Retrieval via Soft Positive Sample CalibrationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.324887333:9(5257-5270)Online publication date: Sep-2023
  • (2023)Diversity Multi-View Clustering With Subspace and NMF-Based Manifold LearningIEEE Access10.1109/ACCESS.2023.326483711(37041-37051)Online publication date: 2023
  • (2022)Multiview Classification with Missing-Views Through Adversarial Representation and Inductive Transfer LearningAdvanced Computing10.1007/978-3-030-95502-1_24(305-317)Online publication date: 8-Feb-2022
  • (2021)Scalable Multi-view Subspace Clustering with Unified AnchorsProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475516(3528-3536)Online publication date: 17-Oct-2021
  • (2021)Consensus guided incomplete multi-view spectral clusteringNeural Networks10.1016/j.neunet.2020.10.014133(207-219)Online publication date: Jan-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media