research-article

Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor

Authors:

Qiangqiang Shen,

Jingyong SuAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 7

Article No.: 103, Pages 1 - 19

https://doi.org/10.1145/3589768

Published: 04 May 2023 Publication History

Abstract

As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association matrix. Although the collaborative cluster structure among basic clusterings can be well discovered by ensemble clustering, most advanced ensemble clustering utilizes the self-representation strategy with the constraint of low-rank to explore a shared consensus representation matrix in multiple views. However, they still encounter two challenges: (1) high computational cost caused by both the matrix inversion operation and singular value decomposition of large-scale square matrices; (2) less considerable attention on high-order correlation attributed to the pursue of the two-dimensional pair-wise relationship matrix. In this article, based on low-rank and sparse decomposition from both matrix and tensor perspectives, we propose two novel multi-view ensemble clustering methods, which tangibly decrease computational complexity. Specifically, our first method utilizes low-rank and sparse matrix decomposition to learn one common co-association matrix, while our last method constructs all co-association matrices into one third-order tensor to investigate the high-order correlation among multiple views by low-rank and sparse tensor decomposition. We adopt the alternating direction method of multipliers to solve two convex models by dividing them into several subproblems with closed-form solution. Experimental results on ten real-world datasets prove the effectiveness and efficiency of the proposed two multi-view ensemble clustering methods by comparing them with other advanced ensemble clustering methods.

References

[1]

Tahani Alqurashi and Wenxin Wang. 2019. Clustering ensemble method. Int. J. Mach. Learn. Cybernet. 10 (2019), 1227–1246.

[2]

Thierry Bouwmans, Necdet Serhat Aybat, and El-hadi Zahzah. 2016. Handbook of Robust Low-rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. CRC Press.

[3]

Thierry Bouwmans, Namrata Vaswani, Paul Rodriguez, René Vidal, and Zhouchen Lin. 2018. Introduction to the issue on robust subspace learning and tracking: Theory, algorithms, and applications. IEEE J. Select. Topics in Signal Process. 12, 6 (2018), 1127–1130.

[4]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122.

Digital Library

[5]

Jian-Feng Cai, Emmanuel J. Candès, and Zuowei Shen. 2010. A singular value thresholding algorithm for matrix completion. SIAM J. Optimiz. 20, 4 (2010), 1956–1982.

[6]

Venkat Chandrasekaran, Pablo A. Parrilo, and Alan S. Willsky. 2010. Latent variable graphical model selection via convex optimization. In Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton’10). IEEE, 1610–1613.

[7]

Yongyong Chen, Yanwen Guo, Yongli Wang, Dong Wang, Chong Peng, and Guoping He. 2017. Denoising of hyperspectral images using nonconvex low rank matrix approximation. IEEE Trans. Geosci. Remote Sens. 55, 9 (2017), 5366–5380.

[8]

Yongyong Chen, Shuqin Wang, Chong Peng, Zhongyun Hua, and Yicong Zhou. 2021. Generalized nonconvex low-rank tensor approximation for multi-view subspace clustering. IEEE Trans. Image Process. 30 (2021), 4022–4035.

Digital Library

[9]

Yongyong Chen, Xiaolin Xiao, and Yicong Zhou. 2019. Low-rank quaternion approximation for color image processing. IEEE Trans. Image Process. 29 (2019), 1426–1439.

[10]

C.-H. Chou, M.-C. Su, and Eugene Lai. 2004. A new cluster validity measure and its application to image compression. Pattern Anal. Appl. 7, 2 (2004), 205–220.

[11]

Inderjit S. Dhillon, Subramanyam Mallela, and Rahul Kumar. 2003. A divisive information theoretic feature clustering algorithm for text classification. J. Mach. Learn. Res. 3 (2003), 1265–1287.

Digital Library

[12]

Xiaoli Zhang Fern and Carla E. Brodley. 2004. Solving cluster ensemble problems by bipartite graph partitioning. In Proceedings of the 21st International Conference on Machine Learning. 36.

Digital Library

[13]

Ana L. N. Fred and Anil K. Jain. 2005. Combining multiple clusterings using evidence accumulation. IEEE Trans. Pattern Anal. Mach. Intell. 27, 6 (2005), 835–850.

Digital Library

[14]

Danfeng Hong, Wei He, Naoto Yokoya, Jing Yao, Lianru Gao, Liangpei Zhang, Jocelyn Chanussot, and Xiaoxiang Zhu. 2021. Interpretable hyperspectral artificial intelligence: When nonconvex modeling meets hyperspectral remote sensing. IEEE Geosci. Remote Sens. Mag. 9, 2 (2021), 52–87.

[15]

Yue Hu and Daniel B. Work. 2020. Robust tensor recovery with fiber outliers for traffic events. ACM Trans. Knowl. Discov. Data 15, 1 (2020), 1–27.

Digital Library

[16]

Misha E. Kilmer, Karen Braman, Ning Hao, and Randy C. Hoover. 2013. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl. 34, 1 (2013), 148–172.

Digital Library

[17]

Hongmin Li, Xiucai Ye, Akira Imakura, and Tetsuya Sakurai. 2020. Ensemble learning for spectral clustering. In Proceedings of the IEEE International Conference on Data Mining (ICDM’20). IEEE, 1094–1099.

[18]

Lu Li, Wei Li, Qian Du, and Ran Tao. 2020. Low-rank and sparse decomposition with mixture of Gaussian for hyperspectral anomaly detection. IEEE Trans. Cybernet. 51, 9 (2020), 4363–4372.

[19]

Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. 2012. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1 (2012), 171–184.

Digital Library

[20]

Hongfu Liu and Yun Fu. 2018. Consensus guided multi-view clustering. ACM Trans. Knowl. Discov. Data 12, 4 (2018), 1–21.

Digital Library

[21]

Hongfu Liu, Tongliang Liu, Junjie Wu, Dacheng Tao, and Yun Fu. 2015. Spectral ensemble clustering. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 715–724.

Digital Library

[22]

Hongfu Liu, Junjie Wu, Tongliang Liu, Dacheng Tao, and Yun Fu. 2017. Spectral ensemble clustering via weighted K-means: Theoretical and practical evidence. IEEE Trans. Knowl. Data Eng. 29, 5 (2017), 1129–1143.

Digital Library

[23]

Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. 2019. Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42, 4 (2019), 925–938.

Digital Library

[24]

Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. 2018. Consistent and specific multi-view subspace clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[25]

Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems. MIT Press, 849–856.

Digital Library

[26]

Alexander Strehl and Joydeep Ghosh. 2002. Cluster ensembles—A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3(Dec.2002), 583–617.

[27]

Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2017. From ensemble clustering to multi-view clustering. In Proceedings of the International Joint Conference on Artificial Intelligence. 2843–2849.

[28]

Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2019. Marginalized multiview ensemble clustering. IEEE Trans. Neural Netw. Learn. Syst. 31, 2 (2019), 600–611.

[29]

Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2019. Robust spectral ensemble clustering via rank minimization. ACM Trans. Knowl. Discov. Data 13, 1 (2019), 1–25.

Digital Library

[30]

Zhiqiang Tao, Hongfu Liu, Sheng Li, and Yun Fu. 2016. Robust spectral ensemble clustering. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 367–376.

Digital Library

[31]

Alexander Topchy, Anil K. Jain, and William Punch. 2003. Combining multiple weak clusterings. In Proceedings of the 3rd IEEE International Conference on Data Mining. IEEE, 331–338.

Digital Library

[32]

Namrata Vaswani, Thierry Bouwmans, Sajid Javed, and Praneeth Narayanamurthy. 2018. Robust subspace learning: Robust PCA, robust subspace tracking, and robust subspace recovery. IEEE Signal Process. Mag. 35, 4 (2018), 32–55.

[33]

Namrata Vaswani, Yuejie Chi, and Thierry Bouwmans. 2018. Rethinking PCA for modern data sets: Theory, algorithms, and applications [scanning the issue]. Proc. IEEE 106, 8 (2018), 1274–1276.

[34]

Haishen Wang, Yan Yang, Hongjun Wang, and Dahai Chen. 2013. Soft-voting clustering ensemble. In Proceedings of the International Workshop on Multiple Classifier Systems. Springer, 307–318.

[35]

Hanrui Wu and Michael K. Ng. 2022. Multiple graphs and low-rank embedding for multi-source heterogeneous domain adaptation. ACM Trans. Knowl. Discov. Data 16, 4 (2022), 1–25.

Digital Library

[36]

Junjie Wu, Hongfu Liu, Hui Xiong, Jie Cao, and Jian Chen. 2014. K-means-based consensus clustering: A unified view. IEEE Trans. Knowl. Data Eng. 27, 1 (2014), 155–169.

[37]

Rongkai Xia, Yan Pan, Lei Du, and Jian Yin. 2014. Robust multi-view spectral clustering via low-rank and sparse decomposition. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, Vol. 28. 367–376.

[38]

Yuan Xie, Bingqian Lin, Yanyun Qu, Cuihua Li, Wensheng Zhang, Lizhuang Ma, Yonggang Wen, and Dacheng Tao. 2020. Joint deep multi-view learning for image clustering. IEEE Trans. Knowl. Data Eng. 33, 11 (2020), 3594–3606.

Digital Library

[39]

Yuan Xie, Jinyan Liu, Yanyun Qu, Dacheng Tao, Wensheng Zhang, Longquan Dai, and Lizhuang Ma. 2020. Robust kernelized multiview self-representation for subspace clustering. IEEE Trans. Neural Netw. Learn. Syst. 32, 2 (2020), 868–881.

[40]

Yuan Xie, Dacheng Tao, Wensheng Zhang, Yan Liu, Lei Zhang, and Yanyun Qu. 2018. On unifying multi-view self-representations for clustering by tensor multi-rank minimization. Int. J. Comput. Vision 126, 11 (2018), 1157–1179.

Digital Library

[41]

Yuan Xie, Wensheng Zhang, Yanyun Qu, Longquan Dai, and Dacheng Tao. 2018. Hyper-Laplacian regularized multilinear multiview self-representations for clustering and semisupervised learning. IEEE Trans. Cybernet. 50, 2 (2018), 572–586.

[42]

Xiyu Yu, Tongliang Liu, Xinchao Wang, and Dacheng Tao. 2017. On compressing deep models by low rank and sparse decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7370–7379.

[43]

Changqing Zhang, Huazhu Fu, Qinghua Hu, Xiaochun Cao, Yuan Xie, Dacheng Tao, and Dong Xu. 2018. Generalized latent multi-view subspace clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42, 1 (2018), 86–99.

Digital Library

[44]

Zhao Zhang, Jiahuan Ren, Sheng Li, Richang Hong, Zhengjun Zha, and Meng Wang. 2019. Robust subspace discovery by block-diagonal adaptive locality-constrained representation. In Proceedings of the 27th ACM International Conference on Multimedia. 1569–1577.

Digital Library

[45]

Di Zhong, HongJiang Zhang, and Shih-Fu Chang. 1996. Clustering methods for video browsing and annotation. In Storage and Retrieval for Still Image and Video Databases IV, Vol. 2670. SPIE, 239–246.

[46]

Jie Zhou, Hongchan Zheng, and Lulu Pan. 2019. Ensemble clustering based on dense representation. Neurocomputing 357 (2019), 66–76.

Digital Library

Cited By

Shen QZhang XWang SLi YLiang YChen Y(2025)Dual Completion Learning for Incomplete Multi-View ClusteringIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34515629:1(455-467)Online publication date: Feb-2025
https://doi.org/10.1109/TETCI.2024.3451562
Hu JShi NKontar RYan H(2025)Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor DataTechnometrics10.1080/00401706.2025.2453206(1-27)Online publication date: 17-Jan-2025
https://doi.org/10.1080/00401706.2025.2453206
Zhang HHuang TZhao XZhang SXie JJiang TNg M(2024)Learnable Transform-Assisted Tensor Decomposition for Spatio-Irregular Multidimensional Data RecoveryACM Transactions on Knowledge Discovery from Data10.1145/3701235Online publication date: 21-Oct-2024
https://doi.org/10.1145/3701235
Show More Cited By

Index Terms

Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
    2. Machine learning algorithms
      1. Ensemble methods
      2. Spectral methods

Recommendations

Multi-view document clustering via ensemble method

Multi-view clustering has become an important extension of ensemble clustering. In multi-view clustering, we apply clustering algorithms on different views of the data to obtain different cluster labels for the same set of objects. These results are ...
Enhanced tensor low-rank representation learning for multi-view clustering
Abstract
Multi-view subspace clustering (MSC), assuming the multi-view data are generated from a latent subspace, has attracted considerable attention in multi-view clustering. To recover the underlying subspace structure, a successful approach ...
Highlights
- Capturing intrinsic high-order correlations underlying multi-view data.
- ...
Nonconvex low-rank and sparse tensor representation for multi-view subspace clustering
Abstract
Multi-view subspace clustering has attracted significant attention due to the popularity of multi-view datasets. The effectiveness of the existing multi-view clustering methods highly depends on the quality of the affinity matrix. To derive a high ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 7

August 2023

319 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3589018

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 May 2023

Online AM: 30 March 2023

Accepted: 26 March 2023

Revised: 27 January 2023

Received: 29 August 2022

Published in TKDD Volume 17, Issue 7

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Guangdong Natural Science Foundation
Shenzhen College Stability Support Plan
Shenzhen Science and Technology Program
Humanities and Social Sciences Foundation of the Ministry of Education of China
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
863
Total Downloads

Downloads (Last 12 months)321
Downloads (Last 6 weeks)30

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shen QZhang XWang SLi YLiang YChen Y(2025)Dual Completion Learning for Incomplete Multi-View ClusteringIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34515629:1(455-467)Online publication date: Feb-2025
https://doi.org/10.1109/TETCI.2024.3451562
Hu JShi NKontar RYan H(2025)Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor DataTechnometrics10.1080/00401706.2025.2453206(1-27)Online publication date: 17-Jan-2025
https://doi.org/10.1080/00401706.2025.2453206
Zhang HHuang TZhao XZhang SXie JJiang TNg M(2024)Learnable Transform-Assisted Tensor Decomposition for Spatio-Irregular Multidimensional Data RecoveryACM Transactions on Knowledge Discovery from Data10.1145/3701235Online publication date: 21-Oct-2024
https://doi.org/10.1145/3701235
Yu XLiu HZhang YLin YZhang C(2024)Hubness-Enabled Clustering and Recovery for Large-Scale Incomplete Multi-View DataACM Transactions on Knowledge Discovery from Data10.1145/369468919:1(1-23)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3694689
Li HZhang LSu KYu W(2024)MICCF: A Mutual Information Constrained Clustering Framework for Learning Clustering-Oriented Feature RepresentationsACM Transactions on Knowledge Discovery from Data10.1145/367240218:8(1-22)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1145/3672402
Chen XDong ZBi JYang KLv JYu Z(2024)A Multi-view Clustering Ensemble via Graph Tensor Learning in Medicine Images2024 IEEE International Conference on Medical Artificial Intelligence (MedAI)10.1109/MedAI62885.2024.00023(123-129)Online publication date: 15-Nov-2024
https://doi.org/10.1109/MedAI62885.2024.00023
Zheng DYu ZChen WZhang WFeng QShi YYang K(2024)Multiview Ensemble Clustering of Hypergraph p-Laplacian Regularization with Weighting and DenoisingInformation Sciences10.1016/j.ins.2024.121187(121187)Online publication date: Jul-2024
https://doi.org/10.1016/j.ins.2024.121187
Akinnubi AAgarwal NAlassad MAjiboye JRokne JWang D(2023)Knowledge Graph Embedding for Topical and Entity Classification in Multi-Source Social Network DataProceedings of the 2023 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining10.1145/3625007.3627315(530-537)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625007.3627315
Zhang ZZhang ZYao JLiu LLi JWu GWu X(2023)Multi-Label Feature Selection Via Adaptive Label Correlation EstimationACM Transactions on Knowledge Discovery from Data10.1145/360456017:9(1-28)Online publication date: 10-Aug-2023
https://dl.acm.org/doi/10.1145/3604560
Pronello NIgnaccolo RIppoliti LFontanella S(2023)Penalized model-based clustering of complex functional dataStatistics and Computing10.1007/s11222-023-10288-233:6Online publication date: 25-Aug-2023
https://dl.acm.org/doi/10.1007/s11222-023-10288-2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents