Skip to main content
SpringerLink
Log in

Adaptive factorization rank selection-based NMF and its application in tumor recognition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The nonnegative matrix factorization (NMF) has been widely used because it can accomplish both feature representation learning and dimension reduction. However, there are two critical and challenging issues affecting the performance of NMF models. One is the selection of matrix factorization rank, while most of the existing methods are based on experiments or experience. For tackling this issue, an adaptive and stable NMF model is constructed based on an adaptive factorization rank selection (AFRS) strategy, which skillfully and simply integrates a row constraint similar to the generalized elastic net. The other is the sensitivity to the initial value of the iteration, which seriously affects the result of matrix factorization. This issue is alleviated by complementing NMF and deep learning each other and avoiding complex network structure. The proposed NMF model is called deep AFRS-NMF model for short, and the corresponding optimization solution, convergence and stability are analyzed. Moreover, the statistical consistency is discussed between the rank obtained by the proposed model and the ideal rank. The performance of the proposed deep AFRS-NMF model is demonstrated by applying in genetic data-based tumor recognition. Experiments show that the factorization rank obtained by the deep AFRS-NMF model is stable and superior to classical and state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791

    Article  MATH  Google Scholar 

  2. Gillis N, Glineur F (2010) Using under approximations for sparse non-negative matrix factorization. Pattern Recogn 43:1676–1687

    Article  MATH  Google Scholar 

  3. Wild S, Curry J, Dougherty A (2004) Improving non-negative matrix factorizations through structured initialization. Pattern Recogn 37:2217–2232

    Article  Google Scholar 

  4. Boutsidis C, Gallopoulos E (2008) SVD based initialization: a head start for non-negative matrix factorization. Pattern Recogn 41:1350–1362

    Article  MATH  Google Scholar 

  5. Zheng CH, Ng TY, Zhang L, Shiu CK, Wang HQ (2011) Tumor classification based on non-negative matrix factorization using gene expression data. IEEE Trans Nanobiosci 10:86–93

    Article  Google Scholar 

  6. Tu D, Chen L, Chen GC, Wu Y, Wang JC (2018) Hierarchical online NMF for detecting and tracking topic hierarchies in a text stream. Pattern Recogn 76:203–214

    Article  Google Scholar 

  7. Cichocki A, Zdunek R (2006) Multilayer non-negative matrix factorization. Electron Lett 42:947–948

    Article  Google Scholar 

  8. Hoyer PO (2002) Non-negative sparse coding. IEEE Workshop on Neural Networks for Signal Processing, vol 0202009. pp 557–565

  9. Miura I, Tachioka Y, Narita T (2016) Multi-channel non-negative matrix factorization with binary mask initialization for automatic speech recognition. J Acoust Soc Am 140:3450–3450

    Article  Google Scholar 

  10. Liu XS, Wang B, Zhang LM (2010) A novel approach for hyperspectral unmixing based on non-negative matrix factorization. In IEEE International geoscience & remote sensing symposium. pp 1289–1292

  11. Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436

    Article  Google Scholar 

  12. Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248

    Article  Google Scholar 

  13. Zhang W, Li R, Deng H et al (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 108:214–224

    Article  Google Scholar 

  14. Suk HI, Lee SW, Shen D et al (2014) Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. Neuroimage 101:569–582

    Article  Google Scholar 

  15. Hinton GE, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  16. Han ZY, Wei BZ, Zheng YJ et al (2017) Breast cancer multi-classification from histopathological images with structured deep learning. Sci Rep 4172:1–10

    Google Scholar 

  17. Bengio Y, Lamblin P, Popovici D et al (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153–160

    Google Scholar 

  18. Salakhutdinov R, Hinton G (2012) An efficient learning procedure for deep Boltzmann machines. Neural Comput 24:1967–2006

    Article  MathSciNet  MATH  Google Scholar 

  19. Sarikaya R, Hinton GE, Deoras A (2014) Application of deep belief networks for natural language understanding. IEEE/ACM Trans Audio Speech Lang Process 22:778–784

    Article  Google Scholar 

  20. Le Roux J, Hershey JR, Weninger F (2015) Deep NMF for speech separation. IEEE International conference on acoustics, speech and signal processing. pp 66–70

  21. Trigeorgis G, Bousmalis K, Zafeiriou S et al (2016) A deep matrix factorization method for learning attribute representations. IEEE Trans Pattern Anal Mach Intell 39:417–429

    Article  Google Scholar 

  22. Yang XH, Wu WM, Chen YM et al (2019) An integrated inverse space sparse representation framework for tumor classification. Pattern Recogn 93:293–311

    Article  Google Scholar 

  23. Xue Y, Tong CS, Chen YCW (2008) Clustering-based initialization for non-negative matrix factorization. Appl Math Comput 205:525–536

    MathSciNet  MATH  Google Scholar 

  24. Yang Z, Zhu Z, Oja E (2010) Automatic rank determination in projective non-negative matrix factorization. In: Proceedings of 9th international conference on latent variable analysis and signal separation. pp 514–521

  25. Said M, Brie D, Djafari AM et al (2006) Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling. IEEE Trans Signal Process 54:4133–4145

    Article  MATH  Google Scholar 

  26. Tibshirani R (2011) Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Ser B (Statistical Methodology) 73:267–288

    Article  MathSciNet  MATH  Google Scholar 

  27. Pan X, Xu Y (2018) A safe reinforced feature screening strategy for lasso based on feasible solutions. Inf Sci 477:132–147

    Article  MathSciNet  MATH  Google Scholar 

  28. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Statistical Methodology) 67:768

    Article  MathSciNet  MATH  Google Scholar 

  29. Xu Y, Tian Y, Pan X et al (2019) E-ENDPP: a safe feature selection rule for speeding up elastic net. Appl Intell 49:592–604

    Article  Google Scholar 

  30. Gao Y, Church PG (2005) Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21:3970–3975

    Article  Google Scholar 

  31. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30:1145–1159

    Article  Google Scholar 

  32. Alon U, Barkai N, Notterman DA et al (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci 96:6745–6750

    Article  Google Scholar 

  33. Gan B, Zheng CH, Zhang J et al (2014) Sparse representation for tumor classification based on feature extraction using latent low-rank representation. Biomed Res Int 10:63–68

    Google Scholar 

  34. Zheng CH, Zhang L, Ng TY et al (2011) Metasample-based sparse representation for tumor classification. IEEE/ACM Trans Comput Biol Bioinf 8:1273–1282

    Article  Google Scholar 

  35. Liu JX, Xu Y, Zheng CH et al (2015) RPCA-based tumor classification using gene expression data. IEEE/ACM Trans Comput Biol Bioinf 12:964–970

    Article  Google Scholar 

  36. Xiao YH, Chen L, Li D (2018) A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming. Math Program Comput 10:533–555

    Article  MathSciNet  MATH  Google Scholar 

  37. Chen L, Huang JZ (2012) Sparse reduced-rank regression for simultaneous dimension reduction and variable selection. J Am Stat Assoc 107:1533–1545

    Article  MathSciNet  MATH  Google Scholar 

  38. Xin X, Hu J, Liu L (2017) On the oracle property of a generalized adaptive elastic-net for multivariate linear regression with a diverging number of parameters. J Multivar Anal 162:16–31

    Article  MathSciNet  MATH  Google Scholar 

  39. Wright J, Ganesh A, Zhou Z et al (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227

    Article  Google Scholar 

  40. Yang XH, Liu F, Tian L et al (2018) Pseudo-full-space representation based classification for robust face recognition. Signal Process Image Commun 60:64–78

    Article  Google Scholar 

  41. Veer LJV, Dai H, Vijver MJVD et al (2001) Expression profiling predicts poor outcome of disease in young breast cancer patients. Eur J Cancer 37:271–271

    Article  Google Scholar 

  42. Tamayo P (2002) Diffuse large B-cell lymphoma outcome prediction by gene expression profiling and supervised machine learning. Nat Med 8:68–74

    Article  Google Scholar 

  43. Armstrong SA, Staunton JE, Silverman LB et al (2002) MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat Genet 30:41–47

    Article  Google Scholar 

  44. Van't Veer LJ, Dai H, Van De Vijver MJ et al (2002) Gene expression profiling predicts clinical outcome of breast cancer. Nature 415:530–536

    Article  Google Scholar 

  45. Deng W, Hu J, Guo J (2012) Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans Pattern Anal Mach Intell 34:1864–1870

    Article  Google Scholar 

  46. Deng H, Runger G (2013) Gene selection with guided regularized random forest. Pattern Recogn 46:3483–3489

    Article  Google Scholar 

  47. García V, Sánchez JS (2015) Mapping microarray gene expression data into dissimilarity spaces for tumor classification. Inf Sci 294:362–375

    Article  MathSciNet  Google Scholar 

  48. Dettling M, Bühlmann P (2004) BagBoosting for tumor classification with gene expression data. Bioinformatics 20:1061–1069

    Article  Google Scholar 

  49. Ruiz R, Riquelme JC, Ruiz JSA (2006) Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recogn 39:2383–2392

    Article  Google Scholar 

  50. Younsi R, Bagnall A (2016) Ensembles of random sphere cover classifiers. Pattern Recogn 49:213–225

    Article  Google Scholar 

  51. Gan B, Zheng CH, Liu JX (2016) Metasample-based robust sparse representation for tumor classification. Engineering 5:78–83

    Article  Google Scholar 

  52. Hong JH, Cho SB (2009) Gene boosting for cancer classification based on gene expression profiles. Pattern Recogn 42:1761–1767

    Article  MATH  Google Scholar 

  53. Piao Y, Piao M, Park K et al (2012) An ensemble correlation-based gene selection algorithm for cancer classification with gene expression data. Bioinformatics 28:3306–3315

    Article  Google Scholar 

  54. Zheng D, Jia J, Fang X, et al (2017) Main and interaction effects selection for quadratic discriminant analysis via penalized linear regression. arXiv:1702.04570

  55. Fan Y, Kong Y, Li D et al (2015) Innovated interaction screening for high-dimensional nonlinear classification. Ann Stat 43:1243–1272

    MathSciNet  MATH  Google Scholar 

  56. Jiang BY, Chen ZQ, Leng CL (2020) Dynamic linear discriminant analysis in high dimensional space. Bernoulli 26:1234–1268

    Article  MathSciNet  MATH  Google Scholar 

  57. Su Q, Wang YN, Jiang XB, et al (2017) A cancer gene selection algorithm based on the KS test and CFS. BioMed Res Int 2017:1645619

Download references

Acknowledgements

The authors would like to thank https://tumorgenome.nih.gov/ for their breast datasets. We also thank Prof. Bingsheng He and Yunhai Xiao for their optimization suggestion. This work was supported in part by Natural Science Foundations of China (41771375, 11571025, 11701144, 62003004), Natural Science Foundation of Beijing (1182002), Open Fund of Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education (IPIU2019010), Natural Science Foundations of Henan (202102310087, 202300410066, 212102310305, 202102210125).

Author information

Authors and Affiliations

  1. Henan Engineering Research Center for Artificial Intelligence Theory and Algorithms, School of Mathematics and Statistics, Henan University, Kaifeng, 475004, China

    Xiaohui Yang, Xin Xin & Limin Su

  2. School of Computer Science and Technology, Xidian University, Xi’an, 710071, China

    Wenming Wu

  3. College of Applied Sciences, Beijing University of Technology, Beijing, 100000, China

    Liugen Xue

Authors
  1. Xiaohui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenming Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Limin Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liugen Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xin Xin or Liugen Xue.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Wu, W., Xin, X. et al. Adaptive factorization rank selection-based NMF and its application in tumor recognition. Int. J. Mach. Learn. & Cyber. 12, 2673–2691 (2021). https://doi.org/10.1007/s13042-021-01353-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01353-1

Keywords

Search

Navigation