skip to main content
10.1145/3581783.3611733acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

High-order Complementarity Induced Fast Multi-View Clustering with Enhanced Tensor Rank Minimization

Published: 27 October 2023 Publication History

Abstract

Recently, tensor-based multi-view clustering methods have achieved promising results, primarily benefited from their superior ability in exploring high-order consistent information among views. Despite significant progress, these methods inevitably suffer from several drawbacks: 1) Extremely high computational complexity restricts their feasibility for large-scale data sets. 2) Prevalently adopted tensor rank approximations (e.g., Tensor Nuclear Norm (TNN)) tend to under-penalize small singular values, resulting in noise residuals. 3) Tensor structure is rarely utilized for high-order complementarity investigation. In light of this, we propose High-order Complementarity Induced Fast Multi-View Clustering with Enhanced Tensor Rank Minimization (CFMVC-ETR). Specifically, two sets of representation matrices are learned from original multi-view data via the matrix factorization mechanism with a group of base matrices, which are further reconstructed into the consistent tensor and the complementary tensor, respectively. Subsequently, a novel Enhanced Tensor Rank is imposed on the consistent tensor, which is a tighter approximation of the tensor rank and is more noisy-robust to explore the high-order consistency. Meanwhile, a tensor-level constraint termed Tensorial Exclusive Regularization is proposed on the complementary tensor to enhance the view-specific feature and well capture the high-order complementarity. Moreover, we adopt a concatenation-fusion approach to integrate these two parts, deriving a discriminative unified embedding for the clustering task. We solve CFMVC-ETR by an efficient algorithm with good convergence. Extensive experiments on nine challenging data sets demonstrate the superiority of the proposed method.

References

[1]
Michael W Berry, Murray Browne, Amy N Langville, V Paul Pauca, and Robert J Plemmons. 2007. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis, Vol. 52, 1 (2007), 155--173.
[2]
Thomas Blumensath. 2014. Sparse matrix decompositions for clustering. In 2014 22nd European Signal Processing Conference (EUSIPCO). IEEE, 1163--1167.
[3]
Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. 2015. Diversity-induced multi-view subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 586--594.
[4]
Man-Sheng Chen, Chang-Dong Wang, and Jian-Huang Lai. 2023. Low-Rank tensor based proximity learning for multi-View clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 5 (2023), 5076--5090.
[5]
Yongyong Chen, Xiaolin Xiao, Chong Peng, Guangming Lu, and Yicong Zhou. 2021. Low-rank tensor graph learning for multi-view subspace clustering. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 1 (2021), 92--104.
[6]
James W Cooley and John W Tukey. 1965. An algorithm for the machine calculation of complex Fourier series. Math. Comp., Vol. 19, 90 (1965), 297--301.
[7]
Chris Ding, Xiaofeng He, and Horst D Simon. 2005. On the equivalence of nonnegative matrix factorization and spectral clustering. In Proceedings of the 2005 SIAM International Conference on Data Mining. SIAM, 606--610.
[8]
Chris Ding, Tao Li, Wei Peng, and Haesun Park. 2006. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 126--135.
[9]
Quanxue Gao, Wei Xia, Zhizhen Wan, Deyan Xie, and Pu Zhang. 2020. Tensor-SVD based graph learning for multi-view subspace clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 3930--3937.
[10]
Donald Geman and Chengda Yang. 1995. Nonlinear image recovery with half-quadratic regularization. IEEE Transactions on Image Processing, Vol. 4, 7 (1995), 932--946.
[11]
Jipeng Guo, Yanfeng Sun, Junbin Gao, Yongli Hu, and Baocai Yin. 2023. Logarithmic schatten-p norm minimization for tensorial multi-View subspace clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 3 (2023), 3396--3410.
[12]
Shudong Huang, Yixi Liu, Yazhou Ren, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. 2022a. Learning smooth representation for multi-view subspace clustering. In Proceedings of the 30th ACM International Conference on Multimedia. 3421--3429.
[13]
Shudong Huang, Yixi Liu, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. 2022b. Multi-View subspace clustering by joint measuring of consistency and diversity. IEEE Transactions on Knowledge and Data Engineering (2022), 1--12.
[14]
Zhao Kang, Wangtao Zhou, Zhitong Zhao, Junming Shao, Meng Han, and Zenglin Xu. 2020. Large-scale multi-view subspace clustering in linear time. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 4412--4419.
[15]
Lusi Li and Haibo He. 2020. Bipartite graph based multi-view clustering. IEEE Transactions on knowledge and Data Engineering, Vol. 34, 7 (2020), 3111--3125.
[16]
Xuelong Li, Han Zhang, Rong Wang, and Feiping Nie. 2020. Multiview clustering: A scalable and parameter-free bipartite graph fusion method. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 1 (2020), 330--344.
[17]
Youwei Liang, Dong Huang, Chang-Dong Wang, and S Yu Philip. 2022. Multi-view graph learning by joint modeling of consistency and inconsistency. IEEE Transactions on Neural Networks and Nearning Systems (2022), 1--15.
[18]
Zhouchen Lin, Risheng Liu, and Zhixun Su. 2011. Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems, Vol. 24 (2011), 1--9.
[19]
Jiyuan Liu, Xinwang Liu, Yuexiang Yang, Li Liu, Siqi Wang, Weixuan Liang, and Jiangyong Shi. 2021. One-pass multi-view clustering for large-scale data. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 12344--12353.
[20]
Suyuan Liu, Siwei Wang, Pei Zhang, Kai Xu, Xinwang Liu, Changwang Zhang, and Feng Gao. 2022. Efficient one-pass multi-view subspace clustering with consensus anchors. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7576--7584.
[21]
Yongli Liu, Xiaoqin Zhang, Guiying Tang, and Di Wang. 2019. Multi-view subspace clustering based on tensor Schatten-p norm. In IEEE International Conference on Big Data (Big Data). IEEE, 5048--5055.
[22]
Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. 2016. Tensor robust principal component analysis: exact recovery of corrupted low-rank tensors via convex optimization. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5249--5257.
[23]
Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. 2019. Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, 4 (2019), 925--938.
[24]
Feiping Nie, Shenfei Pei, Rong Wang, and Xuelong Li. 2020. Fast clustering with co-clustering via discrete non-negative matrix factorization for image identification. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2073--2077.
[25]
Mengjing Sun, Pei Zhang, Siwei Wang, Sihang Zhou, Wenxuan Tu, Xinwang Liu, En Zhu, and Changjian Wang. 2021. Scalable multi-view subspace clustering with unified anchors. In Proceedings of the 29th ACM International Conference on Multimedia. 3528--3536.
[26]
Xiaoli Sun, Rui Zhu, Ming Yang, Xiujun Zhang, and Yuanyan Tang. 2022. Sliced sparse gradient induced multi-view subspace clustering via tensorial arctangent rank minimization. IEEE Transactions on Knowledge and Data Engineering (2022), 1--14.
[27]
Chang Tang, Zhenglai Li, Jun Wang, Xinwang Liu, Wei Zhang, and En Zhu. 2022. Unified one-step multi-view spectral clustering. IEEE Transactions on Knowledge and Data Engineering (2022), 1--11.
[28]
Yongqiang Tang, Yuan Xie, and Wensheng Zhang. 2023. Affine subspace robust low-rank self-representation: from matrix to tensor. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023), 1--17.
[29]
Pham Dinh Tao and LT Hoai An. 1997. Convex analysis approach to DC programming: theory, algorithms and applications. Acta Mathematica Vietnamica, Vol. 22, 1 (1997), 289--355.
[30]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, Vol. 9, 11 (2008), 2579--2605.
[31]
Ulrike Von Luxburg. 2007. A tutorial on spectral clustering. Statistics and Computing, Vol. 17, 4 (2007), 395--416.
[32]
Xinhang Wan, Jiyuan Liu, Weixuan Liang, Xinwang Liu, Yi Wen, and En Zhu. 2022. Continual Multi-view Clustering. In Proceedings of the 30th ACM International Conference on Multimedia. 3676--3684.
[33]
Xinhang Wan, Xinwang Liu, Jiyuan Liu, Siwei Wang, Yi Wen, Weixuan Liang, En Zhu, Zhe Liu, and Lu Zhou. 2023. Auto-weighted multi-view clustering for large-scale data. arXiv preprint arXiv:2303.01983 (2023), 1--9.
[34]
Jing Wang, Feng Tian, Hongchuan Yu, Chang Hong Liu, Kun Zhan, and Xiao Wang. 2017. Diverse non-negative matrix factorization for multiview data representation. IEEE Transactions on Cybernetics, Vol. 48, 9 (2017), 2620--2632.
[35]
Jie Wen, Zheng Zhang, Zhao Zhang, Lei Zhu, Lunke Fei, Bob Zhang, and Yong Xu. 2021. Unified tensor framework for incomplete multi-view clustering and missing-view inferring. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 10273--10281.
[36]
Jianlong Wu, Zhouchen Lin, and Hongbin Zha. 2019. Essential tensor learning for multi-view spectral clustering. IEEE Transactions on Image Processing, Vol. 28, 12 (2019), 5910--5922.
[37]
Yanan Wu, Tengfei Liang, Songhe Feng, Yi Jin, Gengyu Lyu, Haojun Fei, and Yang Wang. 2023. MetaZSCIL: A meta-learning approach for generalized zero-shot class incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 10408--10416.
[38]
Wei Xia, Quanxue Gao, Qianqian Wang, Xinbo Gao, Chris Ding, and Dacheng Tao. 2023. Tensorized bipartite graph learning for multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 4 (2023), 5187--5202.
[39]
Yuan Xie, Dacheng Tao, Wensheng Zhang, Yan Liu, Lei Zhang, and Yanyun Qu. 2018. On unifying multi-view self-representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision, Vol. 126, 11 (2018), 1157--1179.
[40]
Weiqing Yan, Jindong Xu, Jinglei Liu, Guanghui Yue, and Chang Tang. 2022. Bipartite graph-based discriminative feature learning for multi-view clustering. In Proceedings of the 30th ACM International Conference on Multimedia. 3403--3411.
[41]
Haizhou Yang, Quanxue Gao, Wei Xia, Ming Yang, and Xinbo Gao. 2022. Multiview spectral clustering with bipartite graph. IEEE Transactions on Image Processing, Vol. 31 (2022), 3591--3605.
[42]
Kun Zhan, Changqing Zhang, Junpeng Guan, and Junsheng Wang. 2017. Graph learning for multi-view clustering. IEEE Transactions on Cybernetics, Vol. 48, 10 (2017), 2887--2895.
[43]
Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, and Xiaochun Cao. 2015. Low-rank tensor constrained multiview subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1582--1590.
[44]
Chen Zhang, Siwei Wang, Jiyuan Liu, Sihang Zhou, Pei Zhang, Xinwang Liu, En Zhu, and Changwang Zhang. 2021b. Multi-view clustering via deep matrix factorization and partition alignment. In Proceedings of the 29th ACM International Conference on Multimedia. 4156--4164.
[45]
Tiejian Zhang, Xinwang Liu, En Zhu, Sihang Zhou, and Zhibin Dong. 2022. Efficient anchor learning-based multi-view clustering--a late fusion method. In Proceedings of the 30th ACM International Conference on Multimedia. 3685--3693.
[46]
Yi Zhang, Xinwang Liu, Siwei Wang, Jiyuan Liu, Sisi Dai, and En Zhu. 2021a. One-stage incomplete multi-view clustering via late fusion. In Proceedings of the 29th ACM International Conference on Multimedia. 2717--2725.
[47]
Handong Zhao, Zhengming Ding, and Yun Fu. 2017. Multi-view clustering via deep matrix factorization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. 2921--2927.
[48]
Pan Zhou, Canyi Lu, Jiashi Feng, Zhouchen Lin, and Shuicheng Yan. 2019. Tensor low-rank representation for data recovery and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, 5 (2019), 1718--1732.

Cited By

View all
  • (2024)Scalable Multi-view Spectral Clustering Based on Spectral Perturbation TheoryProceedings of the ACM Turing Award Celebration Conference - China 202410.1145/3674399.3674434(92-99)Online publication date: 5-Jul-2024
  • (2024)Multiview Tensor Spectral Clustering via Co-RegularizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338682846:10(6795-6808)Online publication date: 9-Apr-2024
  • (2024)Learnable Tensor Graph Fusion Framework for Natural Image SegmentationIEEE Transactions on Multimedia10.1109/TMM.2024.336068926(7160-7173)Online publication date: 31-Jan-2024

Index Terms

  1. High-order Complementarity Induced Fast Multi-View Clustering with Enhanced Tensor Rank Minimization

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '23: Proceedings of the 31st ACM International Conference on Multimedia
    October 2023
    9913 pages
    ISBN:9798400701085
    DOI:10.1145/3581783
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. high-order complementarity
    2. high-order consistency
    3. matrix factorization
    4. tensor-based multi-view clustering

    Qualifiers

    • Research-article

    Funding Sources

    • The Fundamental Research Funds for the Central Universities

    Conference

    MM '23
    Sponsor:
    MM '23: The 31st ACM International Conference on Multimedia
    October 29 - November 3, 2023
    Ottawa ON, Canada

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)204
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Scalable Multi-view Spectral Clustering Based on Spectral Perturbation TheoryProceedings of the ACM Turing Award Celebration Conference - China 202410.1145/3674399.3674434(92-99)Online publication date: 5-Jul-2024
    • (2024)Multiview Tensor Spectral Clustering via Co-RegularizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338682846:10(6795-6808)Online publication date: 9-Apr-2024
    • (2024)Learnable Tensor Graph Fusion Framework for Natural Image SegmentationIEEE Transactions on Multimedia10.1109/TMM.2024.336068926(7160-7173)Online publication date: 31-Jan-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media