Skip to main content
SpringerLink
Log in

Incomplete multi-view clustering via self-attention networks and feature reconstruction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Over the past few years, numerous deep learning-based methods have been proposed for incomplete multi-view clustering. However, these approaches overlook two crucial issues. First, they focus solely on the global information contained in the latent representations derived from deep networks, neglecting the importance of local focal points. Second, while leveraging consistent or complementary inter-view information for cross-view learning, they disregard the intrinsic relationships among different samples within the same view. To address these concerns, this manuscript presents an original approach: incomplete multi-view clustering based on self-attention networks and feature reconstruction (SNFR). Specifically, SNFR initially employs self-attention networks to emphasize the pivotal information within views, aiming to reduce the inter-view reconstruction loss. Subsequently, an improved entropy weighting method is applied to reconstruct the feature relationships among the diverse samples within the same view, thereby facilitating consistent cross-view information learning. Our proposed method is evaluated on six widely used multi-view datasets through extensive experiments, highlighting its remarkable superiority over the alternative approaches in terms of clustering performance

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Fang U, Li M, Li J, Gao L, Jia T, Zhang Y (2023) A comprehensive survey on multi-view clustering. IEEE Trans Knowl Data Eng

  2. Wang H, Jiang G, Peng J, Deng R, Fu X (2022) Towards adaptive consensus graph: multi-view clustering via graph collaboration. IEEE Trans Multimed

  3. Lu Z, Nie F, Wang R, Li X (2022) A differentiable perspective for multi-view spectral clustering with flexible extension. IEEE Trans Pattern Anal Mach Intell 45(6):7087–7098

    Article  Google Scholar 

  4. Pan B, Li C, Che H (2023) Nonconvex low-rank tensor approximation with graph and consistent regularizations for multi-view subspace learning. Neural Netw 161:638–658

    Article  Google Scholar 

  5. Liu X, Li M, Tang C, Xia J, Xiong J, Liu L, Kloft M, Zhu E (2020) Efficient and effective regularized incomplete multi-view clustering. IEEE Trans Pattern Anal Mach Intell 43(8):2634–2646

    Google Scholar 

  6. Liu C, Wu Z, Wen J, Xu Y, Huang C (2022) Localized sparse incomplete multi-view clustering. IEEE Trans Multimed

  7. Li X-L, Chen M-S, Wang C-D, Lai J-H (2022) Refining graph structure for incomplete multi-view clustering. IEEE Trans Neural Netw Learn Syst

  8. Deng S, Wen J, Liu C, Yan K, Xu G, Xu Y (2023) Projective incomplete multi-view clustering. IEEE Trans Neural Netw Learn Syst

  9. Wen J, Yan K, Zhang Z, Xu Y, Wang J, Fei L, Zhang B (2020) Adaptive graph completion based incomplete multi-view clustering. IEEE Trans Multimed 23:2493–2504

    Article  Google Scholar 

  10. Liu X, Zhu X, Li M, Tang C, Zhu E, Yin J, Gao W (2019) Efficient and effective incomplete multi-view clustering. In: The 9th symposium on educational advances in artificial intelligence (EAAI) vol 33, pp 4392–4399

  11. Liu X, Li M, Tang C, Xia J, Xiong J, Liu L, Kloft M, Zhu E (2020) Efficient and effective regularized incomplete multi-view clustering. IEEE Trans Pattern Anal Mach Intell 43(8):2634–2646

    Google Scholar 

  12. Xu J, Li C, Ren Y, Peng L, Mo Y, Shi X, Zhu X (2022) Deep incomplete multi-view clustering via mining cluster complementarity. In: Thirty-sixth conference on artificial intelligence (AAAI), pp 8761–8769

  13. Xu J, Li C, Peng L, Ren Y, Shi X, Shen HT, Zhu X (2023) Adaptive feature projection with distribution alignment for deep incomplete multi-view clustering. IEEE Trans Image Process 32:1354–1366

    Article  Google Scholar 

  14. Wang Q, Ding Z, Tao Z, Gao Q, Fu Y (2018) Partial multi-view clustering via consistent gan. In: 2018 IEEE international conference on data mining (ICDM) pp 1290–1295

  15. Xu C, Guan Z, Zhao W, Wu H, Niu Y, Ling B (2019) Adversarial incomplete multi-view clustering. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, (IJCAI), 10-16 August 2019, pp 3933–3939

  16. Lin Y, Gou Y, Liu Z, Li B, Lv J, Peng X (2021) Completer: incomplete multi-view clustering via contrastive prediction. In: Conference on computer vision and pattern recognition (CVPR), pp 11174–11183

  17. Lin Y, Gou Y, Liu X, Bai J, Lv J, Peng X (2022) Dual contrastive prediction for incomplete multi-view representation learning. IEEE Trans Pattern Anal Mach Intell

  18. Li J, Zhou G, Qiu Y, Wang Y, Zhang Y, Xie S (2020) Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing 390:108–116

    Article  Google Scholar 

  19. Wen J, Xu Y, Liu H (2018) Incomplete multiview spectral clustering with adaptive graph learning. IEEE Trans Cybern 50(4):1418–1429

    Article  Google Scholar 

  20. Liu X, Zhu X, Li M, Wang L, Zhu E, Liu T, Kloft M, Shen D, Yin J, Gao W (2019) Multiple kernel \( k \) k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell 42(5):1191–1204

    Google Scholar 

  21. Zhuge W, Luo T, Tao H, Hou C, Yi D (2020) Multi-view spectral clustering with incomplete graphs. IEEE Access 8:99820–99831

    Article  Google Scholar 

  22. Sun L, Wen J, Liu C, Fei L, Li L () Balance guided incomplete multi-view spectral clustering. Neural Netw 166, 260–272

  23. Liu X, Zhu X, Li M, Wang L, Tang C, Yin J, Shen D, Wang H, Gao W (2018) Late fusion incomplete multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(10):2410–2423

    Article  Google Scholar 

  24. Hu M, Chen S (2018) Doubly aligned incomplete multi-view clustering. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, (IJCAI) pp 2262–2268

  25. Hu M, Chen S (2019) One-pass incomplete multi-view clustering. The thirty-third conference on artificial intelligence (AAAI) 33:3838–3845

    Article  Google Scholar 

  26. Wang Q, Ding Z, Tao Z, Gao Q, Fu Y (2021) Generative partial multi-view clustering with adaptive fusion and cycle consistency. IEEE Trans Image Process 30:1771–1783

    Article  Google Scholar 

  27. Xu J, Li C, Peng L, Ren Y, Shi X, Shen HT, Zhu X (2023) Adaptive feature projection with distribution alignment for deep incomplete multi-view clustering. IEEE Trans Image Process 32:1354–1366

    Article  Google Scholar 

  28. Khan S, Naseer M, Hayat M, Zamir SW, Khan FS, Shah M (2022) Transformers in vision: a survey. ACM Comput Surv (CSUR) 54(10s):1–41

    Article  Google Scholar 

  29. Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: a survey. Comput Vis Media 8(3):331–368

    Article  Google Scholar 

  30. Zheng X, Sun H, Lu X, Xie W (2022) Rotation-invariant attention network for hyperspectral image classification. IEEE Trans Image Process 31:4251–4265

    Article  Google Scholar 

  31. Liu H, Li W, Xia X-G, Zhang M, Gao C-Z, Tao R (2022) Central attention network for hyperspectral imagery classification. IEEE Trans Neural Netw Learn Syst

  32. Zhu P, Yao X, Wang Y, Cao M, Hui B, Zhao S, Hu Q (2022) Latent heterogeneous graph network for incomplete multi-view learning. arXiv:2208.13669

  33. Liu Y, Wan Y, He L, Peng H, Yu PS (2021) KG-BART: knowledge graph-augmented BART for generative commonsense reasoning. In: The eleventh symposium on educational advances in artificial intelligence (EAAI), pp 6418–6425

  34. Cover TM, Thomas JA (2006) Elements of Information Theory (2, Ed)

  35. Tsai YH, Wu Y, Salakhutdinov R, Morency L (2021) Self-supervised learning from a multi-view perspective. In: 9th international conference on learning representations, ICLR 2021, Virtual Event, Austria, 3-7 May 2021

  36. Lopez R, Regier J, Jordan MI, Yosef N (2018) Information constraints on auto-encoding variational bayes. Adv Neural Inf Process Syst 31

  37. Zhu Y, Tian D, Yan F (2020) Effectiveness of entropy weight method in decision-making. Math Probl Eng 2020:1–5

    Google Scholar 

  38. Huang J, Gong S, Zhu X (2020) Deep semantic clustering by partition confidence maximisation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 8846–8855

  39. Peng X, Zhu H, Feng J, Shen C, Zhang H, Zhou JT (2020) Deep clustering with sample-assignment invariance prior. IEEE Trans Neural Netw Learn Syst 31(11):4857–4868

    Article  MathSciNet  Google Scholar 

  40. Wen J, Liu C, Deng S, Liu Y, Fei L, Yan K, Xu Y (2023) Deep double incomplete multi-view multi-label learning with incomplete labels and missing views. IEEE Trans Neural Netw Learn Syst

  41. Liu S, Wang S, Zhang P, Xu K, Liu X, Zhang C, Gao F (2022) Efficient one-pass multi-view subspace clustering with consensus anchors. In: The twelveth symposium on educational advances in artificial intelligence (EAAI), pp 7576–7584

  42. Peng X, Huang Z, Lv J, Zhu H, Zhou JT (2019) COMIC: multi-view clustering without parameter selection. In: Proceedings of the 36th international conference on machine learning (ICML). Proceedings of Machine Learning Research vol 97, pp 5092–5101

  43. Liu W, Liu L, Feng L, Deng H (2022) Tensorized multi-view clustering via hyper-graph regularization. In: 2022 international joint conference on neural networks (IJCNN), IEEE, pp 1–8

  44. Kim H, Hwang S, Park J, Yun S, Lee J-H, Park B-G (2018) Spiking neural network using synaptic transistors and neuron circuits for pattern recognition with noisy images. IEEE Electron Device Lett 39(4):630–633

  45. Liang N, Yang Z, Li Z, Sun W, Xie S (2020) Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints. Knowl-Based Syst 194:105582

    Article  Google Scholar 

  46. Trosten DJ, Lokse S, Jenssen R, Kampffmeyer M (2021) Reconsidering representation alignment for multi-view clustering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1255–1265

  47. Li S, Jiang Y, Zhou Z (2014) Partial multi-view clustering. In: Proceedings of the twenty-eighth conference on artificial intelligence (AAAI), pp 1968–1974

  48. Zhang C, Liu Y, Fu H (2019) Ae2-nets: autoencoder in autoencoder networks. In: IEEE Conference on Comput Vis Pattern Recognit (CVPR), pp 2577–2585

  49. Zhao H, Liu H, Fu Y (2016) Incomplete multi-modal visual data grouping. In: Proceedings of the Twenty-Fifth international joint conference on artificial intelligence (IJCAI), pp 2392–2398

  50. Wang W, Arora R, Livescu K, Bilmes JA (2015) On deep multi-view representation learning. In: Proceedings of the 32nd international conference on machine learning (ICML) vol 37, pp 1083–1092

  51. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Köpf A, Yang EZ, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library, 8024–8035

  52. Andrew G, Arora R, Bilmes JA, Livescu K (2013) Deep canonical correlation analysis. In: Proceedings of the 30th international conference on machine learning (ICML), vol 28, pp 1247–1255

  53. Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782

    Article  Google Scholar 

  54. Wen J, Zhang Z, Xu Y, Zhang B, Fei L, Liu H (2019) Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In: The Ninth symposium on educational advances in artificial intelligence (EAAI), pp 5393–5400

  55. Wang H, Zong L, Liu B, Yang Y, Zhou W (2019) Spectral perturbation meets incomplete multi-view data. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence (IJCAI) pp 3677–3683

  56. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61772252, the Scientific Research Foundation of the Education Department of Liaoning Province under Grant LJKZ0965, and the Huzhou Science and Technology Plan Project under Grant 2022GZ08 and 2023ZD2004.

Author information

Authors and Affiliations

  1. Huzhou University, College of Information Engineering, Huzhou, 313000, Zhejiang, China

    Yong Zhang, Li Jiang, Da Liu & Wenzhe Liu

  2. Liaoning Normal University, School of Computer Science and Information Technology, Dalian, 116029, Liaoning, China

    Yong Zhang

Authors
  1. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Da Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenzhe Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yong Zhang: Resources, Data curation, Formal analysis. Li Jiang: Conceptualization, Methodology. Da Liu: Software, Writing-originaldraft, Visualization. Wenzhe Liu: Supervision, Investigation.

Corresponding author

Correspondence to Wenzhe Liu.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing finan-cial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Jiang, L., Liu, D. et al. Incomplete multi-view clustering via self-attention networks and feature reconstruction. Appl Intell 54, 2998–3016 (2024). https://doi.org/10.1007/s10489-024-05299-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05299-z

Keywords

Search

Navigation