skip to main content
10.1145/3534678.3539282acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Efficient Orthogonal Multi-view Subspace Clustering

Published: 14 August 2022 Publication History

Abstract

Multi-view subspace clustering targets at clustering data lying in a union of low-dimensional subspaces. Generally, an n X n affinity graph is constructed, on which spectral clustering is then performed to achieve the final clustering. Both graph construction and graph partitioning of spectral clustering suffer from quadratic or even cubic time and space complexity, leading to difficulty in clustering large-scale datasets. Some efforts have recently been made to capture data distribution in multiple views by selecting key anchor bases beforehand with k-means or uniform sampling strategy. Nevertheless, few of them pay attention to the algebraic property of the anchors. How to learn a set of high-quality orthogonal bases in a unified framework, while maintaining its scalability for very large datasets, remains a big challenge. In view of this, we propose an Efficient Orthogonal Multi-view Subspace Clustering (OMSC) model with almost linear complexity. Specifically, the anchor learning, graph construction and partition are jointly modeled in a unified framework. With the mutual enhancement of each other, a more discriminative and flexible anchor representation and cluster indicator can be jointly obtained. An alternate minimizing strategy is developed to deal with the optimization problem, which is proved to have linear time complexity w.r.t. the sample number. Extensive experiments have been conducted to confirm the superiority of the proposed OMSC method. The source codes and data are available at https://github.com/ManshengChen/Code-for-OMSC-master.

Supplemental Material

MP4 File
This is the presentation video of our paper termed Efficient Orthogonal Multi-view Subspace Clustering (OMSC). Within the framework, the anchor learning, graph construction and partition are jointly modeled with almost linear complexity. To the best of our knowledge, it is the first time to jointly consider the orthogonal anchor learning, graph construction and partition in a unified model for large-scale data.

References

[1]
Xiao Cai, Feiping Nie, and Heng Huang. 2013. Multi-View K-Means Clustering on Big Data. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2598--2604.
[2]
Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. 2015a. Diversity-induced multi-view subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 586--594.
[3]
Xiaochun Cao, Changqing Zhang, Chengju Zhou, Huazhu Fu, and Hassan Foroosh. 2015b. Constrained Multi-View Video Face Clustering. IEEE Trans. Image Process., Vol. 24, 11 (2015), 4381--4393.
[4]
Mansheng Chen, Ling Huang, Chang-Dong Wang, Dong Huang, and Jian-Huang Lai. 2021. Relaxed multi-view clustering in latent embedding space. Inf. Fusion, Vol. 68 (2021), 8--21.
[5]
Man-Sheng Chen, Ling Huang, Chang-Dong Wang, and Dong Huang. 2020. Multi-view Clustering in Latent Embedding Space. In The Thirty-Fourth AAAI Conference on Artificial Intelligence .
[6]
Xiaojun Chen, Wenya Sun, Bo Wang, Zhihui Li, Xizhao Wang, and Yunming Ye. 2019. Spectral Clustering of Customer Transaction Data With a Two-Level Subspace Weighting Method. IEEE Trans. Cybernetics, Vol. 49, 9 (2019), 3230--3241.
[7]
C. Ding, Tao Li, Wei Peng, and Haesun Park. 2006. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining .
[8]
Chris H. Q. Ding and Xiaofeng He. 2005. On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering. In Proceedings of the 2005 SIAM International Conference on Data Mining. 606--610.
[9]
Guowang Du, Lihua Zhou, Yudi Yang, Kevin Lü, and Lizhen Wang. 2021. Deep Multiple Auto-Encoder-Based Multi-view Clustering. Data Sci. Eng., Vol. 6, 3 (2021), 323--338.
[10]
Li Fei-Fei, Rob Fergus, and Pietro Perona. 2004. Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories. In IEEE Conference on Computer Vision and Pattern Recognition Workshops. 178.
[11]
Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. 2015. Multi-view subspace clustering. In IEEE International Conference on Computer Vision. 4238--4246.
[12]
Junwei Han, Kun Song, Feiping Nie, and Xuelong Li. 2017. Bilateral k-Means Algorithm for Fast Co-Clustering. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. 1969--1975.
[13]
Chenping Hou, Feiping Nie, Hong Tao, and Dongyun Yi. 2017. Multi-View Unsupervised Feature Selection with Adaptive Similarity and View Weight. IEEE Trans. Knowl. Data Eng., Vol. 29, 9 (2017), 1998--2011.
[14]
Han Hu, Zhouchen Lin, Jianjiang Feng, and Jie Zhou. 2014. Smooth representation clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 3834--3841.
[15]
Dong Huang, Chang-Dong Wang, and Jian-Huang Lai. 2022. Fast Multi-view Clustering via Ensembles: Towards Scalability, Superiority, and Simplicity. CoRR, Vol. abs/2203.11572 (2022).
[16]
Ling Huang, Hong-Yang Chao, and Chang-Dong Wang. 2019. Multi-View Intact Space Clustering. Pattern Recognition, Vol. 86 (2019), 344--353.
[17]
Ling Huang, Chang-Dong Wang, Hongyang Chao, and Philip S. Yu. 2020. MVStream: Multiview Data Stream Clustering. IEEE Trans. Neural Networks Learn. Syst., Vol. 31, 9 (2020), 3482--3496.
[18]
Zhao Kang, Liangjian Wen, Wenyu Chen, and Zenglin Xu. 2019. Low-rank kernel learning for graph-based clustering. Knowledge-Based Systems, Vol. 163 (2019), 510--517.
[19]
Zhao Kang, Xinjia Zhao, Chong Peng, Hongyuan Zhu, Joey Tianyi Zhou, Xi Peng, Wenyu Chen, and Zenglin Xu. 2020 a. Partition level multiview subspace clustering. Neural Networks, Vol. 122 (2020), 279--288.
[20]
Zhao Kang, Wangtao Zhou, Zhitong Zhao, Junming Shao, Meng Han, and Zenglin Xu. 2020 b. Large-Scale Multi-View Subspace Clustering in Linear Time. In The Thirty-Fourth AAAI Conference on Artificial Intelligence. 4412--4419.
[21]
Yeqing Li, Feiping Nie, Heng Huang, and Junzhou Huang. 2015. Large-Scale Multi-View Spectral Clustering via Bipartite Graph. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence . 2750--2756.
[22]
Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. 2013. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 35, 1 (2013), 171--184.
[23]
Xinwang Liu, Xinzhong Zhu, Miaomiao Li, Lei Wang, Chang Tang, Jianping Yin, Dinggang Shen, Huaimin Wang, and Wen Gao. 2019. Late Fusion Incomplete Multi-View Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 41, 10 (2019), 2410--2423.
[24]
Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. 2018. Consistent and specific multi-view subspace clustering. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence .
[25]
Feiping Nie, Guohao Cai, Jing Li, and Xuelong Li. 2018. Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification. IEEE Trans. Image Process., Vol. 27, 3 (2018), 1501--1511.
[26]
Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence . 1881--1887.
[27]
Feiping Nie, Jing Li, and Xuelong Li. 2017. Self-weighted Multiview Clustering with Multiple Graphs. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence . 2564--2570.
[28]
Feiping Nie, Shaojun Shi, and Xuelong Li. 2020. Auto-weighted multi-view co-clustering via fast matrix factorization. Pattern Recognit., Vol. 102 (2020), 107207.
[29]
Mengjing Sun, Pei Zhang, Siwei Wang, Sihang Zhou, Wenxuan Tu, Xinwang Liu, En Zhu, and Changjian Wang. 2021. Scalable Multi-view Subspace Clustering with Unified Anchors. In ACM Multimedia Conference . 3528--3536.
[30]
Shiliang Sun, Liang Mao, Ziang Dong, and Lidan Wu. 2019. Multiview Machine Learning .Springer.
[31]
Chang Tang, Xinzhong Zhu, Xinwang Liu, Miaomiao Li, Pichao Wang, Changqing Zhang, and Lizhe Wang. 2019. Learning a Joint Affinity Graph for Multiview Subspace Clustering. IEEE Trans. Multimedia, Vol. 21, 7 (2019), 1724--1736.
[32]
Chang-Dong Wang, Mansheng Chen, Ling Huang, Jian-Huang Lai, and Philip S. Yu. 2021. Smoothness Regularized Multiview Subspace Clustering With Kernel Learning. IEEE Trans. Neural Networks Learn. Syst., Vol. 32, 11 (2021), 5047--5060.
[33]
Chang-Dong Wang, Jian-Huang Lai, and Philip S. Yu. 2016. Multi-View Clustering Based on Belief Propagation. IEEE Trans. Knowl. Data Eng., Vol. 28, 4 (2016), 1007--1021.
[34]
Hua Wang, Feiping Nie, Heng Huang, and Fillia Makedon. 2011. Fast Nonnegative Matrix Tri-Factorization for Large-Scale Data Co-Clustering. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence . 1553--1558.
[35]
Hao Wang, Yan Yang, and Bing Liu. 2020. GMC: Graph-Based Multi-View Clustering. IEEE Trans. Knowl. Data Eng., Vol. 32, 6 (2020), 1116--1129.
[36]
Jing Wang, Feng Tian, Hongchuan Yu, Chang Hong Liu, Kun Zhan, and Xiao Wang. 2018. Diverse Non-Negative Matrix Factorization for Multiview Data Representation. IEEE Trans. Cybern., Vol. 48, 9 (2018), 2620--2632.
[37]
Siwei Wang, Xinwang Liu, En Zhu, Chang Tang, Jiyuan Liu, Jingtao Hu, Jingyuan Xia, and Jianping Yin. 2019 b. Multi-view Clustering via Late Fusion Alignment Maximization. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence . 3778--3784.
[38]
Siwei Wang, Xinwang Liu, Xinzhong Zhu, Pei Zhang, Yi Zhang, Feng Gao, and En Zhu. 2022. Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance. IEEE Trans. Image Process., Vol. 31 (2022), 556--568.
[39]
X. Wang, X. Guo, Z. Lei, C. Zhang, and S. Z. Li. 2017. Exclusivity-Consistency Regularized Multi-view Subspace Clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 1--9.
[40]
Xiaobo Wang, Zhen Lei, Xiaojie Guo, Changqing Zhang, Hailin Shi, and Stan Z. Li. 2019 a. Multi-view subspace clustering with intactness-aware similarity. Pattern Recognit., Vol. 88 (2019), 50--63.
[41]
Baoyuan Wu, Yifan Zhang, Bao-Gang Hu, and Qiang Ji. 2013. Constrained Clustering and Its Application to Face Clustering in Videos. In IEEE Conference on Computer Vision and Pattern Recognition. 3507--3514.
[42]
Yuan Xie, Dacheng Tao, Wensheng Zhang, Yan Liu, Lei Zhang, and Yanyun Qu. 2018. On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization. Int. J. Comput. Vis., Vol. 126, 11 (2018), 1157--1179.
[43]
Chang Xu, Dacheng Tao, and Chao Xu. 2013. A Survey on Multi-view Learning. CoRR, Vol. abs/1304.5634 (2013). arxiv: 1304.5634
[44]
Ben Yang, Xuetao Zhang, Feiping Nie, Fei Wang, Weizhong Yu, and Rong Wang. 2021. Fast Multi-View Clustering via Nonnegative and Orthogonal Factorization. IEEE Trans. Image Process., Vol. 30 (2021), 2575--2586.
[45]
Hong Yu, Jia Tang, Guoyin Wang, and Xinbo Gao. 2021. A Novel Multi-View Clustering Method for Unknown Mapping Relationships Between Cross-View Samples. In The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Feida Zhu, Beng Chin Ooi, and Chunyan Miao (Eds.). ACM, 2075--2083.
[46]
Changqing Zhang, Huazhu Fu, Qinghua Hu, Xiaochun Cao, Yuan Xie, Dacheng Tao, and Dong Xu. 2020. Generalized Latent Multi-View Subspace Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 42, 1 (2020), 86--99.
[47]
Changqing Zhang, Huazhu Fu, Si Liu, Guangcan Liu, and Xiaochun Cao. 2015. Low-Rank Tensor Constrained Multiview Subspace Clustering. In IEEE International Conference on Computer Vision. 1582--1590.
[48]
Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent multi-view subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 4279--4287.

Cited By

View all
  • (2025)Angular Reconstructive Discrete Embedding With Fusion Similarity for Multi-View ClusteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348790737:1(45-59)Online publication date: Jan-2025
  • (2025)Partition-level fusion induced multi-view Subspace Clustering with Tensorial Geman RankNeural Networks10.1016/j.neunet.2024.106849182(106849)Online publication date: Mar-2025
  • (2025)Adaptive multi-view subspace clustering algorithm based on representative features and redundant instancesNeurocomputing10.1016/j.neucom.2024.128839620(128839)Online publication date: Mar-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2022
5033 pages
ISBN:9781450393850
DOI:10.1145/3534678
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. large-scale
  2. multi-view clustering
  3. orthogonal bases
  4. partition

Qualifiers

  • Research-article

Funding Sources

Conference

KDD '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)310
  • Downloads (Last 6 weeks)24
Reflects downloads up to 28 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Angular Reconstructive Discrete Embedding With Fusion Similarity for Multi-View ClusteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348790737:1(45-59)Online publication date: Jan-2025
  • (2025)Partition-level fusion induced multi-view Subspace Clustering with Tensorial Geman RankNeural Networks10.1016/j.neunet.2024.106849182(106849)Online publication date: Mar-2025
  • (2025)Adaptive multi-view subspace clustering algorithm based on representative features and redundant instancesNeurocomputing10.1016/j.neucom.2024.128839620(128839)Online publication date: Mar-2025
  • (2025)Scalable unpaired multi-view clustering with Bipartite Graph MatchingInformation Fusion10.1016/j.inffus.2024.102786116(102786)Online publication date: May-2025
  • (2024)EDISONProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692728(16548-16567)Online publication date: 21-Jul-2024
  • (2024)Self-Supervised, Multi-View, Semantics-Aware Anchor ClusteringElectronics10.3390/electronics1323478213:23(4782)Online publication date: 4-Dec-2024
  • (2024)Continual multi-view clustering with consistent anchor guidanceProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/601(5434-5442)Online publication date: 3-Aug-2024
  • (2024)Self-supervised weighted information bottleneck for multi-view clusteringProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/513(4643-4650)Online publication date: 3-Aug-2024
  • (2024)Fast unpaired multi-view clusteringProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/496(4488-4496)Online publication date: 3-Aug-2024
  • (2024)Dual contrastive graph-level clustering with multiple cluster perspectives alignmentProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/417(3770-3779)Online publication date: 3-Aug-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media