Skip to main content
SpringerLink
Log in

Robust self-tuning multi-view clustering

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Previous methods of multi-view clustering focused on the improvement of clustering effectiveness by detecting common information of all views and individual information for every view, but they ignore the following issues, i.e., the initialization sensitivity, the cluster number determination, and the influence of outliers. However, either single-view clustering or multi-view clustering often suffers from above issues. In this paper, we propose a robust self-tuning multi-view clustering to introduce a sum-of-norm loss function to explore the issue of initialization sensitivity, design a sum-of-norm regularization to automatically determine the cluster number, and employ robust statistics techniques to reduce influence of outliers. Furthermore, we propose an effective alternating optimization method to solve the resulting objective function and then theoretically prove its convergence. Experimental results on both synthetic and real data sets demonstrated that our proposed multi-view clustering method outperformed the state-of-the-art clustering methods, in terms of four clustering evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6

Similar content being viewed by others

Notes

  1. In this paper, we changed the original feature matrix Xv to \(\hat {\mathbf {X}}^{v}\), but still consider the issue of outlier influence reduction based on two reasons: 1) the samples in \(\hat {\mathbf {X}}^{v}\) transferred from the outliers in Xv still influence the construction of clustering models, and 2) the samples in \(\hat {\mathbf {X}}^{v}\) still have diversity, i.e., different samples have different importance for the clustering model.

  2. https://elki-project.github.io/.

References

  1. Arthur, D., Vassilvitskii, S.: The advantages of careful seeding k-means+ +. In: SODA, pages 1027–1035 (2007)

  2. Bahmani, B., Moseley, B., Vattani, A., Kumar, R., Vassilvitskii, S.: Scalable k-means+ +. VLDB 5(7), 622–633 (2012)

    Google Scholar 

  3. Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996)

    Article  Google Scholar 

  4. Branco, D.P.P., Carvalho, F.D.A.T.: Fuzzy clustering of multi-view relational data with pairwise constraints. In: IEEE International Conference on Fuzzy Systems, pp. 1–6 (2017)

  5. Brito, M.R., Chavez, E.L., Quiroz, A.J., Yukich, J.E.: Connectivity of the mutual k-nearest-neighbor graph in clustering and outlier detection. Stat. Probabl. Lett. 35(1), 33–42 (1997)

    Article  MathSciNet  Google Scholar 

  6. Cao, X., Zhang, C., Fu, H., Si, L., Zhang, H.: Diversity-induced multi-view subspace clustering. In: CVPR, pp. 586–594 (2015)

  7. Emre Celebi, M., Kingravi, H.A, Vela, P.A: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)

    Article  Google Scholar 

  8. Charbonnier, P., Blanc-Fėraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Transations Image Processing 6(2), 298–311 (1997)

    Article  Google Scholar 

  9. Chi, E.C, Lange, K.: Splitting methods for convex clustering. J. Comput. Graph. Stat. 24(4), 994–1013 (2015)

    Article  MathSciNet  Google Scholar 

  10. Cleuziou, G., Exbrayat, M., Martin, L., Sublemontier, J.-H.: Cofkm: A centralized method for multiple-view clustering. In: ICDM, pages 752–757 (2009)

  11. Comaniciu, D., shift, Peter Meer.: Mean A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  12. Ding, C., He, X., Simon, H.D: On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, pp. 606–610 (2005)

  13. Ding, Z., Shao, M., Fu, Y.: Robust multi-view representation: A unified perspective from multi-view learning to domain adaption. In: IJCAI, pp. 5434–5440 (2018)

  14. Elhamifar, E., Vidal, R.: Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)

    Article  Google Scholar 

  15. Fahad, A., Alshatri, N., Tari, Z., Alamri, A.: A survey of clustering algorithms for big data Taxonomy and empirical analysis. IEEE Trans. Emerg. Topics Comput. 2(3), 267–279 (2014)

    Article  Google Scholar 

  16. Fränti, P., Sieranoja, S.: How much can k-means be improved by using better initialization and repeats? Pattern Recogn. 93, 95–112 (2019)

    Article  Google Scholar 

  17. Frey, B.J, Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)

    Article  MathSciNet  Google Scholar 

  18. Gan, J., Peng, Z., Zhu, X., Hu, R., Ma, J., Wu, G.: Brain functional connectivity analysis based on multi-graph fusion. Med. Image Anal. https://doi.org/10.1016/j.media.2021.102057 (2021)

  19. He, R., Zheng, Wei-Shi, Tan, T., Sun, Z.: Half-quadratic-based iterative minimization for robust sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 261–275 (2014)

    Article  Google Scholar 

  20. Hocking, T.D., Joulin, A., Bach, F., Vert, J.-P.: Clusterpath an algorithm for clustering using convex fusion penalties. In: ICML, p. 1 (2011)

  21. Hu, R., Peng, Z., Zhu, X., Gan, J., Zhu, Y., Ma, J., Wu, G.: Multi-band brain network analysis for functional neuroimaging biomarker identification. IEEE Trans. Med. Imaging. https://doi.org/10.1109/TMI.2021.3099641 (2021)

  22. Hu, R., Zhu, X., Zhu, Y., Gan, J.: Robust svm with adaptive graph learning. World Wide Web 23, 1945–1968 (2020)

    Article  Google Scholar 

  23. Yizhang, J., Fu-Lai, C., Shitong, W., Zhaohong, D., Jun, W., Pengjiang, Q.: Collaborative fuzzy clustering from multiple weighted views. IEEE Trans. Cybern. 45(4), 688–701 (2015)

    Article  Google Scholar 

  24. Lakshmi, M.A, Daniel, G.V., Srinivasa Rao, D.: Initial centroids for k-means using nearest neighbors and feature means. In: Soft Computing and Signal Processing, pp. 27–34. Springer (2019)

  25. Long, B., Yu, P.S., Zhang, Z.: A general model for multiple view unsupervised learning. In: SDM, pp. 822–833 (2008)

  26. Nie, F., Cai, G., Li, X.: Multi-view clustering and semi-supervised classification with adaptive neighbours. In: AAAI, pp. 2408–2414 (2017)

  27. Nikolova, M., Chan, R.H.: The equivalence of half-quadratic minimization and the gradient linearization iteration. IEEE Trans. Image Process. 16(6), 1623–7 (2007)

    Article  MathSciNet  Google Scholar 

  28. Nikolova, M., Ng, M.K.: Analysis of half-quadratic minimization methods for signal and image recovery. SIAM J. Sci. Comput. 27(3), 937–966 (2005)

    Article  MathSciNet  Google Scholar 

  29. Rodriguez, A., Laio, A.: Clustering by fast search and find of density peaks. Science 344(6191), 1492–1496 (2014)

    Article  Google Scholar 

  30. Shah, S.A., Koltun, V.: Robust continuous clustering. Proc. Natl. Acad. Sci. 114(37), 9814–9819 (2017)

    Article  Google Scholar 

  31. Shen, H.T., Zhu, Y., Zheng, W., Zhu, X.: Half-quadratic minimization for unsupervised feature selection on incomplete data. IEEE Trans. Neural Netw. Learn. Syst. https://doi.org/10.1109/TNNLS.2020.3009632 (2020)

  32. Strehl, A., Ghosh, J.: Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3(Dec), 583–617 (2002)

    MathSciNet  MATH  Google Scholar 

  33. Tao, Z., Liu, H., Li, S., Ding, Z., Fu, Y.: From ensemble clustering to multi-view clustering. In: IJCAI (2017)

  34. Wang, B., Yang, Y., Xu, X., Hanjalic, A., Shen, H.T.: Adversarial cross-modal retrieval. In: ACM MM, pp. 154–162 (2017)

  35. Xia, R., Pan, Y., Du, L., Yin, J.: Robust multi-view spectral clustering via low-rank and sparse decomposition. In: AAAI (2014)

  36. Xu, C., Tao, D., Xu, C.: Multi-view self-paced learning for clustering. In: IJCAI, pp. 3974–3980 (2015)

  37. Zhe, X., Guorong, L., Shuhui, W., Jun, H., Weigang, Z., Qingming, H.: Beyond global fusion: A group-aware fusion approach for multi-view image clustering. Inf. Sci. (2019)

  38. Zhan, K., Nie, F., Wang, J., Yi, Y.: Multiview consensus graph clustering. IEEE Trans. Image Process. 28(3), 1261–1270 (2019)

    Article  MathSciNet  Google Scholar 

  39. Zhang, Z., Li, L., Shen, F., Shen, H.T., Shao, L.: Binary multi-view clustering. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1774–1782 (2018)

    Article  Google Scholar 

  40. Zheng, Q., Zhu, J., Li, Z., Pang, S., Wang, J.: Feature concatenation multi-view subspace clustering. arXiv:1901.10657 (2019)

  41. Zhou, H., Liu, Y.: Accurate integration of multi-view range images using k-means clustering. Pattern Recogn. 41(1), 152–175 (2008)

    Article  MathSciNet  Google Scholar 

  42. Zhu, L., Huang, Z., Li, Z., Xie, L., Shen, H.T.: Exploring auxiliary context: Discrete semantic transfer hashing for scalable image retrieval. IEEE Trans. Neural Netw. Learn. Syst. 29(11), 5264–5276 (2018)

    Article  MathSciNet  Google Scholar 

  43. Zhu, M., Martinez, A.M: Subclass discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1274–1286 (2006)

    Article  Google Scholar 

  44. Zhu, X., Gan, J., Guangquan, L u, Li, J., Zhang, S.: Spectral clustering via half-quadratic optimization. World Wide Web 23, 1969–1988 (2020)

    Article  Google Scholar 

  45. Zhu, X., Li, H., Shen, H.T., Zhang, Z., Ji, Y., Fan, Y.: Fusing functional connectivity with network nodal information for sparse network pattern learning of functional brain networks. Inform. Fusion 75, 131–139 (2021)

    Article  Google Scholar 

  46. Xiaofeng, Z., Bin, S., Feng, S., Yanbo, C., Rongyao, H., Jiangzhang, G., Wenhai, Z., Man, L., Liye, W., Yaozong, G., et al.: Joint prediction and time estimation of covid-19 developing severe symptoms using chest ct scan. Med. Image Anal. 67, 101824 (2021)

    Article  Google Scholar 

  47. Zhu, X., Yang, J., Zhang, C., Zhang, S.: Efficient utilization of missing data in cost-sensitive learning. IEEE Trans. Knowl. Data Eng. https://doi.org/10.1109/TKDE.2019.2956530 (2019)

  48. Zhu, X., Zhang, S., He, W., Hu, R., Lei, C., Zhu, P.: One-step multi-view spectral clustering. IEEE Trans. Knowl. Data Eng. 31(10), 2022–2034 (2018)

    Article  Google Scholar 

  49. Zhu, X., Zhang, S., Zhu, Y., Zhu, P., Gao, Y.: Unsupervised spectral feature selection with dynamic hyper-graph learning. IEEE Trans. Knowl. Data Eng. https://doi.org/10.1109/TKDE.2020.3017250 (2020)

  50. Zhu, X., Zhu, Y., Zheng, W.: Spectral rotation for deep one-step clustering. Pattern Recognit. https://doi.org/10.1016/j.patcog.2019.107175 (2019)

  51. Zhu, Y., Zhu, X., Zheng, W.: Robust multi-view learning via half-quadratic minimization. In: IJCAI, pp. 3278–3284 (2018)

Download references

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (Grant No: 61876046); the Guangxi ”Bagui” Teams for Innovation and Research; and the Sichuan Science and Technology Program (Grants No: 2018GZDZX0032 and 2019YFG0535).

Author information

Authors and Affiliations

  1. Guangxi Academy of Sciences, Nanning, 540000, China

    Changan Yuan, Yonghua Zhu, Wei Zheng & Xiaofeng Zhu

  2. Nanning Normal University, Nanning, 540001, China

    Zhi Zhong

  3. Guangxi Normal University, Guilin, 541004, China

    Xiaofeng Zhu

Authors
  1. Changan Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonghua Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yonghua Zhu or Xiaofeng Zhu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Web Intelligence =Artificial Intelligence in the Connected World

Guest Editors: Yuefeng Li, Amit Sheth, Athena Vakali, and Xiaohui Tao

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, C., Zhu, Y., Zhong, Z. et al. Robust self-tuning multi-view clustering. World Wide Web 25, 489–512 (2022). https://doi.org/10.1007/s11280-021-00945-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00945-9

Keywords

Search

Navigation