Skip to main content
SpringerLink
Log in

Coupled block diagonal regularization for multi-view subspace clustering

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

The object of multi-view subspace clustering is to uncover the latent low-dimensional structure by segmenting a collection of high-dimensional multi-source data into their corresponding subspaces. Existing methods imposed various constraints on the affinity matrix and/or the cluster labels to promote segmentation accuracy, and demonstrated effectiveness in some applications. However, the previous constraints are inefficient to ensure the ideal discriminative capability of the corresponding method. In this paper, we propose to learn view-specific affinity matrices and a common cluster indicator matrix jointly in a unified minimization problem, in which the affinity matrices and the cluster indicator matrix can guide each other to facilitate the final segmentation. To enforce the ideal discrimination, we use a block diagonal inducing regularity to constrain the affinity matrices as well as the cluster indicator matrix. Such coupled regularities are double insurances to promote clustering accuracy. We call it Coupled Block Diagonal Regularized Multi-view Subspace Clustering (CBDMSC). Based on the alternative minimization method, an algorithm is proposed to solve the new model. We evaluate our method by several metrics and compare it with several state-of-the-art related methods on some commonly used datasets. The results demonstrate that our method outperforms the state-of-the-art methods in the vast majority of metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html.

  2. http://cvc.yale.edu/projects/yalefaces/yalefaces.html.

  3. http://www.uk.research.att.com/facedatabase.html.

  4. http://www.cs.columbia.edu/CAVE/software/softlib/.

  5. http://mlg.ucd.ie/datasets/.

  6. http://archive.ics.uci.edu/ml/datasets.html.

References

  • Boyd S, Parikh N, Chu E, Peleato B, Eckstein J, et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning 3(1):1–122

  • Cai X, Nie F, Huang H, Kamangar F (2011) Heterogeneous image feature integration via multi-modal spectral clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 1977–1984

  • Cai X, Nie F, Huang H (2013) Multi-view k-means clustering on big data. In: Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI

  • Cao X, Zhang C, Fu H, Liu S, Zhang H (2015) Diversity-induced multi-view subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 586–594

  • Chaudhuri K, Kakade SM, Livescu K, Sridharan K (2009) Multi-view clustering via canonical correlation analysis. In: Proceedings of the International Conference on Machine Learning, ICML, 129–136

  • Chen H, Wang W, Feng X (2018) Structured sparse subspace clustering with grouping-effect-within-cluster. Pattern Recognition 83:107–118

    Article  Google Scholar 

  • Chen MS, Huang L, Wang CD, Huang D, Philip SY (2020) Multiview subspace clustering with grouping effect. IEEE Transactions on Cybernetics

  • Dattorro J (2010) Convex optimization & Euclidean distance geometry. Lulu. com

  • Deng Z, Choi KS, Jiang Y, Wang J, Wang S (2016) A survey on soft subspace clustering. Information sciences 348:84–106

    Article  MathSciNet  Google Scholar 

  • Domeniconi C, Gunopulos D, Ma S, Yan B, Al-Razgan M, Papadopoulos D (2007) Locally adaptive metrics for clustering high dimensional data. Data Mining and Knowledge Discovery 14(1):63–97

    Article  MathSciNet  Google Scholar 

  • Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2790–2797

  • Elhamifar E, Vidal R (2013) Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(11):2765–2781

    Article  Google Scholar 

  • Friedman JH, Meulman JJ (2004) Clustering objects on subsets of attributes (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 66(4):815–849

    Article  MathSciNet  Google Scholar 

  • Gao H, Nie F, Li X, Huang H (2015) Multi-view subspace clustering. In: Proceedings of the IEEE International Conference on Computer Vision, IEEE, 4238–4246

  • Guo J, Yin W, Sun Y, Hu Y (2019) Multi-view subspace clustering with block diagonal representation. IEEE Access 7:84829–84838

    Article  Google Scholar 

  • Huang S, Kang Z, Xu Z (2018) Self-weighted multi-view clustering with soft capped norm. Knowledge-Based Systems 158:1–8

    Article  Google Scholar 

  • Kang Z, Shi G, Huang S, Chen W, Pu X, Zhou JT, Xu Z (2020) Multi-graph fusion for multi-view spectral clustering. Knowledge-Based Systems 189:105102

    Article  Google Scholar 

  • Kang Z, Zhao X, Peng C, Zhu H, Zhou JT, Peng X, Chen W, Xu Z (2020) Partition level multiview subspace clustering. Neural Networks 122:279–288

    Article  Google Scholar 

  • Lades M, Vorbruggen JC, Buhmann J, Lange J, Von Der Malsburg C, Wurtz RP, Konen W (1993) Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42(3):300–311

    Article  Google Scholar 

  • Li CG, You C, Vidal R (2017) Structured sparse subspace clustering: A joint affinity learning and subspace clustering framework. IEEE Transactions on Image Processing 26(6):2988–3001

    Article  MathSciNet  Google Scholar 

  • Liang W, Zhou S, Xiong J, Liu X, Wang S, Zhu E, Cai Z, Xu X (2020) Multi-view spectral clustering with high-order optimal neighborhood laplacian matrix. IEEE Transactions on Knowledge and Data Engineering

  • Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055

  • Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2012) Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(1):171–184

    Article  Google Scholar 

  • Liu J, Liu X, Xiong J, Liao Q, Zhou S, Wang S, Yang Y (2020) Optimal neighborhood multiple kernel clustering with adaptive local kernels. IEEE Transactions on Knowledge and Data Engineering

  • Liu J, Liu X, Yang Y, Guo X, Kloft M, He L (2021) Multiview subspace clustering via co-training robust data representation. IEEE Transactions on Neural Networks and Learning Systems

  • Liu X, Dou Y, Yin J, Wang L, Zhu E (2016) Multiple kernel k-means clustering with matrix-induced regularization. In: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 30

  • Lu C, Feng J, Lin Z, Yan S (2013) Correlation adaptive subspace segmentation by trace lasso. In: Proceedings of the IEEE International Conference on Computer Vision, IEEE, 1345–1352

  • Lu C, Feng J, Lin Z, Mei T, Yan S (2018) Subspace clustering by block diagonal representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(2):487–501

    Article  Google Scholar 

  • Luo S, Zhang C, Zhang W, Cao X (2018) Consistent and specific multi-view subspace clustering. In: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 32

  • Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987

    Article  Google Scholar 

  • Qi L, Shi Y, Wang H, Yang W, Gao Y (2016) Multi-view subspace clustering via a global low-rank affinity matrix. In: Proceedings of the International Conference on Intelligent Data Engineering and Automated Learning, 321–331

  • Rai P, Trivedi A, Daumé H, DuVall SL (2010) Multiview clustering with incomplete views. In: Proceedings of the NIPS Workshop on Machine Learning for Social Computing, Citeseer

  • Ren Y, Domeniconi C, Zhang G, Yu G (2014) A weighted adaptive mean shift clustering algorithm. In: Proceedings of the SIAM International Conference on Data Mining, SIAM, 794–802

  • Shalev-Shwartz Shai (2014) Understanding machine learning : from theory to algorithms

  • Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8):888–905

    Article  Google Scholar 

  • Tao H, Hou C, Qian Y, Zhu J, Yi D (2020) Latent complete row space recovery for multi-view subspace clustering. IEEE Transactions on Image Processing 29:8083–8096

    Article  MathSciNet  Google Scholar 

  • Vidal R (2011) Subspace clustering. IEEE Signal Processing Magazine 28(2):52–68

    Article  Google Scholar 

  • Wang J, Tian F, Wang X, Yu H, Liu C, Yang L (????) Multi-component nonnegative matrix factorization. In: Proceedings of the International Joint Conferences on Artificial Intelligence

  • Wang W, Yang C, Chen H, Feng X (2018a) Unified discriminative and coherent semi-supervised subspace clustering. IEEE Transactions on Image Processing 27(5):2461–2470

    Article  MathSciNet  Google Scholar 

  • Wang X, Guo X, Lei Z, Zhang C, Li SZ (2017b) Exclusivity-consistency regularized multi-view subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 923–931

  • Wang X, Lei Z, Guo X, Zhang C, Shi H, Li SZ (2019) Multi-view subspace clustering with intactness-aware similarity. Pattern Recognition 88:50–63

    Article  Google Scholar 

  • Wang Y, Lin X, Wu L, Zhang W, Zhang Q, Huang X (2015) Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Transactions on Image Processing 24(11):3939–3949

    Article  MathSciNet  Google Scholar 

  • Wang Y, Wu L, Lin X, Gao J (2018b) Multiview spectral clustering via structured low-rank matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 29(10):4833–4843

    Article  Google Scholar 

  • Wei L, Ji F, Liu H, Zhou R, Zhu C, Zhang X (2021) Subspace clustering via structured sparse relation representation. IEEE Transactions on Neural Networks and Learning Systems

  • Xia G, Sun H, Feng L, Zhang G, Liu Y (2017) Human motion segmentation via robust kernel sparse subspace clustering. IEEE Transactions on Image Processing 27(1):135–150

    Article  MathSciNet  Google Scholar 

  • Xia R, Pan Y, Du L, Yin J (2014) Robust multi-view spectral clustering via low-rank and sparse decomposition. In: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 2149–2155

  • Xie D, Zhang X, Gao Q, Han J, Xiao S, Gao X (2019) Multiview clustering by joint latent representation and similarity learning. IEEE transactions on cybernetics 50(11):4848–4854

    Article  Google Scholar 

  • Xie X, Guo X, Liu G, Wang J (2018) Implicit block diagonal low-rank representation. IEEE Transactions on Image Processing 27(1):477–489

    Article  MathSciNet  Google Scholar 

  • Xie Y, Liu J, Qu Y, Tao D, Zhang W, Dai L, Ma L (2020) Robust kernelized multiview self-representation for subspace clustering. IEEE transactions on neural networks and learning systems 32(2):868–881

    Article  MathSciNet  Google Scholar 

  • Xu J, Yu M, Shao L, Zuo W, Meng D, Zhang L, Zhang D (2019) Scaled simplex representation for subspace clustering. IEEE Transactions on Cybernetics 51(3):1493–1505

    Article  Google Scholar 

  • Yin Q, Wu S, He R, Wang L (2015) Multi-view clustering via pairwise sparse subspace representation. Neurocomputing 156:12–21

    Article  Google Scholar 

  • Yin Q, Wu S, Wang L (2015b) Incomplete multi-view clustering via subspace learning. In: Proceedings of the ACM International on Conference on Information and Knowledge Management, ACM, 383–392

  • You C, Li C, Robinson DP, Vidal R (2018) Scalable exemplar-based subspace clustering on class-imbalanced data. In: Proceedings of the European Conference on Computer Vision, ECCV, 67–83

  • Zhang C, Fu H, Liu S, Liu G, Cao X (2015) Low-rank tensor constrained multiview subspace clustering. In: Proceedings of the IEEE International Conference on Computer Vision, IEEE, 1582–1590

  • Zhang C, Hu Q, Fu H, Zhu P, Cao X (2017a) Latent multi-view subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 4279–4287

  • Zhang C, Fu H, Hu Q, Cao X, Xie Y, Tao D, Xu D (2018) Generalized latent multi-view subspace clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(1):86–99

    Article  Google Scholar 

  • Zhang GY, Zhou YR, Wang CD, Huang D, He XY (2021) Joint representation learning for multi-view subspace clustering. Expert Systems with Applications 166:113913

    Article  Google Scholar 

  • Zhang P, Liu X, Xiong J, Zhou S, Zhao W, Zhu E, Cai Z (2020) Consensus one-step multi-view subspace clustering. IEEE Transactions on Knowledge and Data Engineering

  • Zhang Q, Liu Y, Zhu S, Han J (2017) Salient object detection based on super-pixel clustering and unified low-rank representation. Computer Vision and Image Understanding 161:51–64

    Article  Google Scholar 

  • Zheng Q, Zhu J, Li Z, Pang S, Wang J, Chen L (2020) Consistent and complementary graph regularized multi-view subspace clustering. arXiv preprint arXiv:2004.03106

  • Zhou T, Zhang C, Gong C, Bhaskar H, Yang J (2018) Multiview latent space learning with feature redundancy minimization. IEEE Iransactions on Cybernetics 50(4):1655–1668

    Article  Google Scholar 

  • Zhou T, Zhang C, Peng X, Bhaskar H, Yang J (2019) Dual shared-specific multiview subspace clustering. IEEE transactions on cybernetics 50(8):3517–3530

    Article  Google Scholar 

  • Zhu P, Hui B, Zhang C, Du D, Wen L, Hu Q (2019) Multi-view deep subspace clustering networks. arXiv preprint arXiv:1908.01978

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their considerations and suggestions.

Author information

Authors and Affiliations

  1. College of Science, Zhongyuan University of Technology, Zhengzhou, 450007, China

    Huazhu Chen

  2. College of Mathematics and Statistics, Shenzhen University, Shenzhen, 518060, China

    Weiwei Wang

  3. School of Mathematics and Statistics, Henan University, Kaifeng, 475004, China

    Shousheng Luo

Authors
  1. Huazhu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shousheng Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiwei Wang.

Additional information

Responsible editor: Srinivasan Parthasarathy.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China (61972264), the Natural Science Foundation of Henan Province (212300410320), the Natural Science Foundation of Guangdong Province (2019A1515010894), Natural Science Foundation of Shenzhen (20200807165235002), Program for Science and Technology Development of Henan Province (192102310181,212102310305), Funding for Young Backbone Teachers of Universities in Henan Province (2021GGJS026).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Wang, W. & Luo, S. Coupled block diagonal regularization for multi-view subspace clustering. Data Min Knowl Disc 36, 1787–1814 (2022). https://doi.org/10.1007/s10618-022-00852-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-022-00852-1

Keywords

Search

Navigation