Skip to main content
Log in

Orthogonal Dual Graph-Regularized Nonnegative Matrix Factorization for Co-Clustering

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Nonnegative Matrix Factorization (NMF) has received great attention in the era of big data, owing to its roles in efficiently reducing data dimension and producing feature-based data representation. In this paper, we first propose two new NMF optimization models, called an orthogonal dual graph regularized nonnegative matrix factorization (ODGNMF) method and its modified version: an orthogonal dual graph regularized nonnegative matrix tri-factorization (ODGNMTF) method. Compared with the existing models, our models can preserve the geometrical structures of data manifold and feature manifold by constructing two graphs, and ensure the orthogonality of factor matrices such that they have better NMF performance. Then, two efficient algorithms are developed to solve the models, and the convergence theory of the algorithms is established. Numerical tests by applying our algorithms to mine randomly generated data sets and well-known public databases demonstrate that ODGNMF and ODGNMTF have better numerical performance than the state-of-the-art algorithms in view of computational cost, robustness, sensitivity and sparseness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability Statement

The data related with the findings of this study are available from the corresponding author upon reasonable requests.

References

  1. Zhang, L., Liu, Z., Pu, J., et al.: Adaptive graph regularized nonnegative matrix factorization for data representation. Appl. Intell. 50, 438–447 (2020). https://doi.org/10.1007/s10489-019-01539-9

    Article  Google Scholar 

  2. Gao, Z., Wang, Y., Wu, Q., et al.: Graph regularized L2,1-nonnegative matrix factorization for miRNA-disease association prediction. BMC Bioinform. 21, 61 (2020). https://doi.org/10.1186/s12859-020-3409-x

    Article  Google Scholar 

  3. Wan, Z., Tang, J., Ren, L., et al.: Optimization techniques to deeply mine the transcriptomic profile of the sub-genomes in hybrid fish lineage. Front. Genet. 10, 911 (2019). https://doi.org/10.3389/fgene.2019.00911

    Article  Google Scholar 

  4. Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374, 20150202 (2016). https://doi.org/10.1098/rsta.2015.0202

    Article  MathSciNet  MATH  Google Scholar 

  5. Comon, P., Jutten, C.: Handbook of Blind Source Separation: Independent Component Analysis and Applications. Academic Press, Cambridge (2010)

    Google Scholar 

  6. Lieven, D.L., Bart, D.M., Joos, V.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 1253–1278 (2000). https://doi.org/10.1137/S0895479896305696

    Article  MathSciNet  MATH  Google Scholar 

  7. Wang, H., Zheng, C., Zhao, X.: jNMFMA: a joint non-negative matrix factorization meta-analysis of transcriptomics data. Bioinformatics 31, 572–580 (2015). https://doi.org/10.1093/bioinformatics/btu679

    Article  Google Scholar 

  8. Shang, R., Song, J., Jiao, L., et al.: Double feature selection algorithm based on low-rank sparse non-negative matrix factorization. Int. J. Mach. Learn. Cybernet. 11, 1891–1908 (2020). https://doi.org/10.1007/s13042-020-01079-6

    Article  Google Scholar 

  9. Belachew, M.T.: Efficient algorithm for sparse symmetric nonnegative matrix factorization. Pattern Recogn. Lett. 125, 735–741 (2019). https://doi.org/10.1016/j.patrec.2019.07.026

    Article  Google Scholar 

  10. Tosyali, A., Kim, J., Choi, J., et al.: Regularized asymmetric nonnegative matrix factorization for clustering in directed networks. Pattern Recogn. Lett. 125, 750–757 (2019). https://doi.org/10.1016/j.patrec.2019.07.005

    Article  Google Scholar 

  11. Peng, S., Ser, W., Chen, B., et al.: Robust nonnegative matrix factorization with local coordinate constraint for image clustering. Eng. Appl. Artif. Intell. 88, 103354 (2020). https://doi.org/10.1016/j.engappai.2019.103354

    Article  Google Scholar 

  12. Chen, G., Xu, C., Wang, J., et al.: Graph regularization weighted nonnegative matrix factorization for link prediction in weighted complex network. Neurocomputing 369, 50–60 (2020). https://doi.org/10.1016/j.neucom.2019.08.068

    Article  Google Scholar 

  13. Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)

    MathSciNet  MATH  Google Scholar 

  14. Cai, D., He, X., Han, J., et al.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1548–1560 (2011). https://doi.org/10.1109/TPAMI.2010.231

    Article  Google Scholar 

  15. Li, X., Chen, M., Wang, Q.: Discrimination-aware projected matrix factorization. IEEE Trans. Knowl. Data Eng. 32, 809–814 (2020). https://doi.org/10.1109/TKDE.2019.2936855

    Article  Google Scholar 

  16. Wang, Q., He, X., Jiang, X., et al.: Robust Bi-stochastic graph regularized matrix factorization for data clustering. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.3007673

    Article  Google Scholar 

  17. Gu, Q., Zhou, J.: Co-clustering on manifolds. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 359–368 (2009). https://doi.org/10.1145/1557019.1557063

  18. Shang, F., Jiao, L.C., Wang, F.: Graph dual regularization non-negative matrix factorization for co-clustering. Pattern Recogn. 45, 2237–2250 (2012). https://doi.org/10.1016/j.patcog.2011.12.015

    Article  MATH  Google Scholar 

  19. Wang, Q., Chen, M., Nie, F., et al.: Detecting coherent groups in crowd scenes by multi-view clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42, 46–58 (2020). https://doi.org/10.1109/TPAMI.2018.2875002

    Article  Google Scholar 

  20. Sun, J., Wang, Z., Sun, F., et al.: Sparse dual graph-regularized NMF for image co-clustering. Neurocomputing 316, 156–165 (2018). https://doi.org/10.1016/j.neucom.2018.07.062

    Article  Google Scholar 

  21. Ding, C., Li, T., Peng, W., et al.: Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 126–135 (2006). https://doi.org/10.1145/1150402.1150420

  22. Yoo, J., Choi, S.: Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds. Inf. Process. Manage. 46, 559–570 (2010). https://doi.org/10.1016/j.ipm.2009.12.007

    Article  Google Scholar 

  23. Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 269–274 (2001). https://doi.org/10.1145/502512.502550

  24. Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. Proceedings of the 9h ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 89–98 (2003). https://doi.org/10.1145/956750.956764

  25. Wang, S., Chang, T., Cui, Y., et al.: Clustering by orthogonal non-negative matrix factorization: a sequential non-convex penalty approach. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5576–5580 (2019). https://doi.org/10.1109/ICASSP.2019.8683466

  26. He, P., Xu, X., Ding, J., et al.: Low-rank nonnegative matrix factorization on Stiefel manifold. Inf. Sci. 514, 131–148 (2020). https://doi.org/10.1016/j.ins.2019.12.004

    Article  MathSciNet  MATH  Google Scholar 

  27. Abe, H., Yadohisa, H.: Orthogonal nonnegative matrix tri-factorization based on Tweedie distributions. Adv. Data Anal. Classif. 13, 825–853 (2019). https://doi.org/10.1007/s11634-018-0348-8

    Article  MathSciNet  MATH  Google Scholar 

  28. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998). https://doi.org/10.1137/S0895479895290954

    Article  MathSciNet  MATH  Google Scholar 

  29. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999). https://doi.org/10.1038/44565

    Article  MATH  Google Scholar 

  30. Stiefel, E.: Richtungsfelder und Fernparallelismus in n-dimensionalen Mannigfaltigkeiten. Commentarii Mathematici Helvetici 8, 305–353 (1935–1936)

  31. Lv, J., Deng, S., Wan, Z.: An efficient single-parameter scaling memoryless Broyden–Fletcher–Goldfarb–Shanno algorithm for solving large scale unconstrained optimization problems. IEEE Access 8, 85664–85674 (2020). https://doi.org/10.1109/ACCESS.2020.2992340

    Article  Google Scholar 

  32. Li, T., Wan, Z.: New adaptive Barzilar–Borwein step size and its application in solving large scale optimization problems. The ANZIAM J. 61, 76–98 (2019). https://doi.org/10.1017/S1446181118000263

    Article  MathSciNet  MATH  Google Scholar 

  33. Guo, J., Wan, Z.: A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing. Math. Probl. Eng. 2019, 5261830 (2019). https://doi.org/10.1155/2019/5261830

    Article  MathSciNet  MATH  Google Scholar 

  34. Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implications. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 601–602 (2005). https://doi.org/10.1145/1076034.1076148

  35. Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Comput. Stat. Data Anal. 52, 3913–3927 (2008). https://doi.org/10.1016/j.csda.2008.01.011

    Article  MathSciNet  MATH  Google Scholar 

  36. Cai, D., He, X., Han, J.: Document clustering using locality preserving indexing. IEEE Trans. Knowl. Data Eng. 17, 1624–1637 (2005). https://doi.org/10.1109/TKDE.2005.198

    Article  Google Scholar 

  37. Huang, S., Wan, Z., Zhang, J.: An extended nonmonotone line search technique for large-scale unconstrained optimization. J. Comput. Appl. Math. 330, 586–604 (2018). https://doi.org/10.1016/j.cam.2017.09.026

    Article  MathSciNet  MATH  Google Scholar 

  38. Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 606–610 (2005). https://doi.org/10.1137/1.9781611972757.70

  39. Sun, J., Cai, X., Sun, F., et al.: Dual graph-regularized constrained nonnegative matrix factorization for image clustering. KSII Trans. Internet Inf. Syst. 11, 2607–2627 (2017). https://doi.org/10.3837/tiis.2017.05.017

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China (Grant No. 71671190).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhong Wan.

Ethics declarations

Conflict of interest

We declare that all the authors have no conflict of interest about submission and publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, J., Wan, Z. Orthogonal Dual Graph-Regularized Nonnegative Matrix Factorization for Co-Clustering. J Sci Comput 87, 66 (2021). https://doi.org/10.1007/s10915-021-01489-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-021-01489-w

Keywords

Navigation