Skip to main content
SpringerLink
Log in

Multiview nonnegative matrix factorization with dual HSIC constraints for clustering

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

To utilize multiple features for clustering, this paper proposes a novel method named as multiview nonnegative matrix factorization with dual HSIC constraints for clustering. The Hilbert-Schmidt independence criterion (HSIC) is employed to measure the correlation(including linear and nonlinear correlation) between the latent representation of each view and the common ones (representation constraint). The independence among the vectors of the basis matrix for each view (basis constraint) is maximized to pursue the discriminant and informative basis. To maintain the nonlinear structure of multiview data, we directly optimize the kernel of the common representation and make its values of the same neighborhood are larger than the others. We adopt partition entropy to constrain the uniformity level of the its values. A novel iterative update algorithm is designed to seek the optimal solutions. We extensively test the proposed algorithm and several state-of-the-art NMF-based multiview methods on four datasets. The clustering results validate the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2017) Feature selection: a data perspective. ACM Comput Surv 50:1–45

    Google Scholar 

  2. Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20:606–626

    Google Scholar 

  3. Dökeroglu T, Deniz A, Kiziloz HE (2022) A comprehensive survey on recent metaheuristics for feature selection. Neurocomputing 494:269–296

    Google Scholar 

  4. Wang S, Lu J, Gu X, Weyori BA, Yang J-y (2016) Unsupervised discriminant canonical correlation analysis based on spectral clustering. Neurocomputing 171:425–433

    Google Scholar 

  5. Ouadfel S, Abd Elaziz M (2021) A multi-objective gradient optimizer approach-based weighted multi-view clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104480

    Article  Google Scholar 

  6. Liu X, Wang L, Zhu X, Li M, Zhu E, Liu T, Liu L, Dou Y, Yin J (2019) Absent multiple kernel learning algorithms. IEEE Trans Pattern Anal Mach Intell 1–1

  7. Zhang C, Fu H, Liu S, Liu G, Cao X (2015) Low-rank tensor constrained multiview subspace clustering. Proceedings of the IEEE International Conference on Computer Vision 1:1582–1590

  8. Zhang C, Fu H, Hu Q, Cao X, Xie Y, Tao D, Xu D (2020) Generalized latent multi-view subspace clustering. IEEE Trans Pattern Anal Mach Intell 42:86–99

    Google Scholar 

  9. Yin H, Hu W, Li F, Lou J (2021) One-step multi-view spectral clustering by learning common and specific nonnegative embeddings. Int J Mach Learn Cyber 12(7):2121–2134

    Google Scholar 

  10. Mekthanavanh V, Li T, Meng H, Yang Y, Hu J (2019) Social web video clustering based on multi-view clustering via nonnegative matrix factorization. Int J Mach Learn Cyber 10(10):2779–2790

    Google Scholar 

  11. Hotelling H (1936) Relations between two sets of variates. Biometrika 28(3–4):321–377

    MATH  Google Scholar 

  12. Guo X, Wang S, Tie Y, Qi L, Guan L (2018) Joint intermodal and intramodal correlation preservation for semi-paired learning. Pattern Recognit 81:36–49

    Google Scholar 

  13. Su S, Fang X, Yang G, Ge B, Zhu Y (2019) Self-balanced multi-view orthogonality correlation analysis for image feature learning. Infrared Phys Technol 100:44–51

    Google Scholar 

  14. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Google Scholar 

  15. Cao X, Zhang C, Fu H, L S, ZhangH ( 2015) Diversity-induced Multi-view Subspace Clustering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 586– 594. IEEE, Boston, MA, USA

  16. Zhou T, Zhang C, Peng X, Bhaskar H, Yang J (2020) Dual shared-specific multiview subspace clustering. IEEE Trans Cybern 50:3517–3530

    Google Scholar 

  17. Yin M, Gao J, Xie S, Guo Y (2019) Multiview subspace clustering via tensorial t-product representation. IEEE Trans Neural Netw Learn Syst 30(3):851–864

    MathSciNet  Google Scholar 

  18. Kumar A, Rai P, Daumé H. III( 2011) Co-regularized multi-view spectral clustering. In: Proc Adv Neural Inf Process Syst pp. 1413– 1421. Curran Associates Inc., USA

  19. Liu X, Dou Y, Yin J, Wang L, Zhu E ( 2016) Multiple kernel k-means clustering with matrix-induced regularization. In: Proc AAAI Conf Artif Intell pp. 1888– 1894

  20. Zhan K, Zhang C, Guan J, Wang J (2018) Graph learning for multiview clustering. IEEE Trans Cybern 48:2887–2895

    Google Scholar 

  21. Liang Y, Huang D, Wang C-D, Yu PS (2022) Multi-view graph learning by joint modeling of consistency and inconsistency. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3192445

    Article  Google Scholar 

  22. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    MATH  Google Scholar 

  23. Deng C, He X, Han J, Huang TS (2011) Graph regularized non-negative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Google Scholar 

  24. Li X, Cui G, Dong Y (2017) Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans Cybern 47(11):3840–3853

    MathSciNet  Google Scholar 

  25. Huang S, Zhao P, Ren Y, Li T, Xu Z (2018) Self-paced and soft-weighted nonnegative matrix factorization for data representation. Knowl Based Syst. https://doi.org/10.1016/j.knosys.2018.10.003

    Article  Google Scholar 

  26. Yi Y, Wang J, Zhou W, Zheng C, Kong J, Qiao S (2020) Non-negative matrix factorization with locality constrained adaptive graph. IEEE Trans Circuits Syst Video Technol 30:427–441

    Google Scholar 

  27. Peng S, Ser W, Chen B, Sun L, Lin Z (2020) Robust nonnegative matrix factorization with local coordinate constraint for image clustering. Eng Appl Artif Intell. https://doi.org/10.1109/ACCESS.2021.3064631

    Article  Google Scholar 

  28. Pascual-Montano A, Carazo JM, Kochi K, Lehmann D, Pascual-Marqui RD (2006) Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Trans Pattern Anal Mach Intell 28(3):403–415

    Google Scholar 

  29. Guo W (2021) Sparse dual graph-regularized deep nonnegative matrix factorization for image clustering. IEEE Access. https://doi.org/10.1109/ACCESS.2021.3064631

    Article  Google Scholar 

  30. Xing Z, Wen M, Peng J, Feng J (2021) Discriminative semi-supervised non-negative matrix factorization for data clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104289

    Article  Google Scholar 

  31. Zhang K, Zhao X, Peng S (2021) Multiple graph regularized semi-supervised nonnegative matrix factorization with adaptive weights for clustering. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2021.104499

    Article  Google Scholar 

  32. Meng Y, Shang R, Jiao L, Zhang W, Yang S (2018) Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2017.11.008

    Article  Google Scholar 

  33. Wang C, Song X, Zhang J (2018) Graph regularized nonnegative matrix factorization with sample diversity for image representation. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2017.10.018

    Article  Google Scholar 

  34. Ou W, Yu S, Li G, Lu J, Zhang K, Xie G (2016) Multi-view non-negative matrix factorization by patch alignment framework with view consistency. Neurocomputing 204:116–124

    Google Scholar 

  35. Zong L, Zhang X, Zhao L, Yu H, Zhao Q (2017) Multi-view clustering via multi-manifold regularized non-negative matrix factorization. Neural Netw 88:74–89

    MATH  Google Scholar 

  36. Wang X, Zhang T, Gao X (2019) Multiview clustering based on non-negative matrix factorization and pairwise measurements. IEEE Trans Syst Man Cybern 49(9):3333–3346

    Google Scholar 

  37. Wang J, Tian F, Yu H, Liu CH, Zhan K, Wang X (2018) Diverse non-negative matrix factorization for multiview data representation. IEEE Trans Cybern 48(9):2620–2632

    Google Scholar 

  38. Yang Z, Liang N, Yan W, Li Z, Xie S (2020) Uniform distribution non-negative matrix factorization for multiview clustering. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.2984552

    Article  Google Scholar 

  39. Khan GA, Hu J, Li T, Diallo B, Wang H (2022) Multi-view data clustering via non-negative matrix factorization with manifold regularization. Int J Mach Learn Cyber 13(3):677–689

    Google Scholar 

  40. Diallo B, Hu J, Li T, Khan GA, Hussein AS (2022) Multi-view document clustering based on geometrical similarity measurement. Int J Mach Learn Cyber 13(3):663–675

    Google Scholar 

  41. Liu J, Wang C, Gao J, Han J ( 2013) Multi-View Clustering via Joint Nonnegative Matrix Factorization. In: Proceedings of International Conference on Data Mining

  42. Li J, Zhou G, Qiu Y, Wang Y, Zhang Y, Xie S (2020) Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing 390:108–116

    Google Scholar 

  43. Niu D, Jordan MI (2014) Iterative discovery of multiple alternative clustering views. IEEE Trans Pattern Anal Mach Intell 36(7):14

    Google Scholar 

  44. Gretton A, Bousquet O, Smola A, Schölkopf B (2005) Measuring statistical dependence with Hilbert-Schmidt norms. Algorithmic Learning Theory. Springer, Heidelberg, pp 63–77

    Google Scholar 

  45. Duchene J, Leclercq S (1988) An optimal transformation for discriminant and principal component analysis. IEEE Trans Pattern Anal Mach Intell 10(6):978–983

    MATH  Google Scholar 

  46. Wang S, Lu J, Gu X, Du H, Yang J (2019) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189

    MATH  Google Scholar 

  47. Zhao Y, Wang H, Pei J (2021) Deep non-negative matrix factorization architecture based on underlying basis images learning. IEEE Trans Pattern Anal Mac Intell 43:1897–1913

    Google Scholar 

  48. Yang Z, Li Q, Wenyin L, Lv J (2019) Shared multi-view data representation for multi-domain event detection. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2893953

    Article  Google Scholar 

  49. Lee DD, Seung HS( 2009) Algorithms for non-negative matrix factorization. In: Proc Adv Neural Inf Process Syst p. 7

  50. Song L, Smola A, Gretton A, Borgwardt KM ( 2007) A dependence maximization view of clustering. In: Proc Int Conf Mach Learn pp. 815– 822. ACM Press, Corvalis, Oregon

  51. Song L, Smola A, Gretton A, Bedo J, Borgwardt K (2012) Feature selection via dependence maximization. J Mach Learn Res 13(47):1393–1434

    MathSciNet  MATH  Google Scholar 

  52. He X, Yan S, Hu Y, Niyogi P, Zhang H-J (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Machine Intell 27(3):328–340

    Google Scholar 

  53. Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8(37):1027–1061

    MATH  Google Scholar 

  54. Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    MathSciNet  Google Scholar 

  55. Liu X, Zhu X, Li M, Wang L, Zhu E, Liu T, Kloft M, Shen D, Yin J, Gao W (2020) Multiple kernel k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2892416

    Article  Google Scholar 

  56. Li T, Dou Y, Liu X, Zhao Y, Lv Q (2017) Multiple kernel clustering with corrupted kernels. Neurocomputing 267:447–454

    Google Scholar 

  57. Ping L, Zhan G, He Q, Shi Z, Lu K (2007) On defining partition entropy by inequalities. IEEE Trans Inf Theory 53(7):3233–3239

    MathSciNet  MATH  Google Scholar 

  58. Simovici DA, Jaroszewicz S (2002) An axiomatization of partition entropy. IEEE Trans Inf Theory 48(7):2138–2142

    MathSciNet  MATH  Google Scholar 

  59. Greene D, Cunningham P ( 2006) Practical solutions to the problem of diagonal dominance in kernel document clustering. In: Proc Int Conf Mach Learn pp. 377– 384. ACM Press, ???

  60. Wang J, Tian F, Yu H, Liu CH, Zhan K, Wang X (2018) Diverse non-negative matrix factorization for multiview data representation. IEEE Trans Cybern 48:2620–2632

    Google Scholar 

  61. Zhao L, Yang T, Zhang J, Chen Z, Yang Y, Wang ZJ (2021) Co-learning non-negative correlated and uncorrelated features for multi-view data. IEEE Trans Neural Netw Learn Syst 32(4):1486–1496

    MathSciNet  Google Scholar 

  62. Liu J, Liu X, Yang Y, Guo X, Kloft M, He L (2021) Multiview subspace clustering via co-training robust data representation. IEEE Trans Neural Netw Learn Syst 33:1–13

    MathSciNet  Google Scholar 

  63. Zhang C, Wang S, Liu J, Zhou S, Zhang P, Liu X, Zhu E, Zhang C ( 2021) Multi-view clustering via deep matrix factorization and partition alignment. In: In Proceedings of Multimedia, pp. 4156– 4164

  64. Nie F, Cai G, Li X ( 2017) Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours. In: Proceedings of the AAAI Conference on Artificial Intelligence, p. 7

  65. Huang D, Wang C-D, Lai J-H (2018) Locally weighted ensemble clustering. IEEE Trans Cybern 48(5):1460–1473

    Google Scholar 

  66. Zhang G-Y, Zhou Y-R, He X-Y, Wang C-D, Huang D (2020) One-step kernel multi-view subspace clustering. Knowl Based Syst 189:105–126

    Google Scholar 

Download references

Acknowledgements

This work is supported in part by the NSF Foundation of China (No.62276162), by Henan Center for Outstanding Overseas Scientists(GZS2022011), by Key R & D and Promotion Project of Henan Province (No.202102210334), by Henan Province Science Foundation for Youths(No.212300410296), by Henan Provincial Department of Science and Technology Research Project(212102310299), by Natural Science Foundation of China(62276162), by Young Scientists Fund of the Natural Science Foundation of Shanxi, China(202103021223464) and Key Research and Development Plan of Shanxi, China(202102070301019).

Author information

Authors and Affiliations

  1. School of intelligent Engineering, Zhengzhou University of Aeronautics, ZhengZhou, Henan, China

    Sheng Wang, Liyong Chen & Yaowei Sun

  2. School of Big Data, Shanxi University, Taiyuan, Shanxi, China

    Furong Peng

  3. School of Computer Science and Engineer, Nanjing University of Science and Technology, Nanjing, Jiangsu, China

    Jianfeng Lu

Authors
  1. Sheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaowei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Furong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianfeng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sheng Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Chen, L., Sun, Y. et al. Multiview nonnegative matrix factorization with dual HSIC constraints for clustering. Int. J. Mach. Learn. & Cyber. 14, 2007–2022 (2023). https://doi.org/10.1007/s13042-022-01742-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01742-0

Keywords

Search

Navigation