Skip to main content
SpringerLink
Log in

Rank-Adaptive Non-Negative Matrix Factorization

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Dimension reduction is a challenge task in data processing, especially in high-dimensional data processing area. Non-negative matrix factorization (NMF), as a classical dimension reduction method, has a contribution to the parts-based representation for the characteristics of non-negative constraints in the NMF algorithm. In this paper, the NMF algorithm is introduced to extract local features for dimension reduction. Considering the problem of which NMF is required to define the number of the decomposition rank manually, we proposed a rank-adaptive NMF algorithm, in which the affinity propagation (AP) clustering algorithm is introduced to determine adaptively the number of the decomposition rank of NMF. Then, the rank-adaptive NMF algorithm is used to extract features for the original image. After that, a low-dimensional representation of the original image is obtained through the projection from the original images to the feature space. Finally, we used extreme learning machine (ELM) and k-nearest neighbor (KNN) as the classifier to classify those low-dimensional feature representations. The experimental results demonstrate that the decomposition rank determined by the AP clustering algorithm can reflect the characteristics of the original data. When it is combined with the classification algorithm ELM or KNN and applied to handwritten character recognition, the proposed method not only reduces the dimension of original images but also performs well in terms of classification accuracy and time consumption. A new rank-adaptive NMF algorithm is proposed based on the AP clustering algorithm and the original NMF algorithm. According to this algorithm, the low-dimensional representation of the original data can be obtained without any prior knowledge. In addition, the proposed rank-adaptive NMF algorithm combined with the ELM and KNN classification algorithms performs well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. R.E. Bellman, Adaptive control processes: a guided tour, Princeton university press; 2015.

  2. Keogh E, Mueen A. Curse of dimensionality. Ind.Eng.Chem. 2017;29(1):48–53.

    Google Scholar 

  3. Griparis A, Faur D, Datcu M. Dimensionality reduction for visual data mining of earth observation archives. IEEE Geoscience & Remote Sensing Letters. 2016;13(11):1701–5. https://doi.org/10.1109/LGRS.2016.2604919.

    Article  Google Scholar 

  4. Travieso CM, Alonso JB. Special issue on advanced cognitive systems based on nonlinear analysis. Cogn Comput. 2013;5(4):397–8. https://doi.org/10.1007/s12559-013-9237-9.

    Article  Google Scholar 

  5. Dan K. A singularly valuable decomposition: the SVD of a matrix. College Mathematics Journal. 1996;27(1):2–23.

    Article  Google Scholar 

  6. Malkomes G, Brito CEFD, Gomes JPP. A stochastic framework for K-SVD with applications on face recognition. Pattern Analysis & Applications. 2016;20(3):1–10.

    Google Scholar 

  7. Hu Y, Yang M. Face recognition algorithm based on algebraic features of SVD and KL projection. In: International conference on robots and intelligent system 2016. Zhangjiajie: IEEE; 2016. p. 193–6.

    Google Scholar 

  8. Al-Anzi F S, Abuzeina D. Toward an enhanced Arabic text classification using cosine similarity and latent semantic indexing. Journal of King Saud University - Computer and Information Sciences. 2017;29(2):189–195.

    Article  Google Scholar 

  9. Yu D, Chen N, Jiang F, et al. Constrained NMF-based semi-supervised learning for social media spammer detection. Knowl-Based Syst. 2017;125(C):64–73.

    Article  Google Scholar 

  10. Liao Q, Zhang Q. Local coordinate based graph-regularized NMF for image representation. Signal Process. 2016;124:103–14. https://doi.org/10.1016/j.sigpro.2015.09.038.

    Article  Google Scholar 

  11. Sunnydayal KK, Cruces S. An iterative posterior NMF method for speech enhancement in the presence of additive Gaussian noise. Neurocomputing. 2017;230:312–5. https://doi.org/10.1016/j.neucom.2016.12.012.

    Article  Google Scholar 

  12. Cai D, He X, Han J, Huang TS. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis & Machine Intelligence. 2011;33(8):1548–60. https://doi.org/10.1109/TPAMI.2010.231.

    Article  Google Scholar 

  13. Kwon K, Shin JW, Kim NS. NMF-based speech enhancement using bases update. IEEE Signal Processing Letters. 2015;22(4):450–4. https://doi.org/10.1109/LSP.2014.2362556.

    Article  Google Scholar 

  14. Yazawa S, Hamanaka M, Utsuro T. Novel approach to separation of musical signal sources by NMF. In: IEEE International conference on signal processing 2014. HangZhou: IEEE; 2015. p. 610–5.

    Google Scholar 

  15. Du R, Kuang D, Drake B, et al. DC-NMF: nonnegative matrix factorization based on divide-and-conquer for fast clustering and topic modeling. J Glob Optim. 2017;68:1–22.

    Article  Google Scholar 

  16. Vital JPM, Faria DR, Dias G, et al. Combining discriminative spatiotemporal features for daily life activity recognition using wearable motion sensing suit. Pattern Analysis & Applications. 2017;20(4):1179–94.

    Article  Google Scholar 

  17. Ge J, Zhou T, Zhang F, et al. Learning part-based dictionary by sparse NMF for face gender recognition. In: International symposium on computational intelligence and design 2015. Hangzhou: IEEE; 2015. p. 375–378.

    Google Scholar 

  18. Liao Q, Zhang Q. Local coordinate based graph-regularized NMF for image representation. Signal Process. 2016;124:103–14. https://doi.org/10.1016/j.sigpro.2015.09.038.

    Article  Google Scholar 

  19. Han M, Liu B. Ensemble of extreme learning machine for remote sensing image classification. Neurocomputing. 2015;149(PA):65–70.

    Article  Google Scholar 

  20. Brunet JP, Tamayo P, Golub TR, Mesirov JP. Metagenes and molecular pattern discovery using matrix factorization. Proc Natl Acad Sci. 2004;101(12):4164–9. https://doi.org/10.1073/pnas.0308531101.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  21. Li SZ, Hou XW, Zhang HJ, et al. Learning spatially localized, parts-based representation. In: IEEE conference on computer vision and pattern recognition 2001. Kauai: IEEE; 2001. p. 207–212.

    Google Scholar 

  22. Xu W, Liu X, Gong Y. Document clustering based on non-negative matrix factorization. In: Proceedings of the twenty-sixth annual international ACM SIGIR conference on research and development in information retrieval 2003. Toronto: ACM; 2003. p. 267–73.

    Google Scholar 

  23. Lee DD, Seung HS. Learning the parts of objects by non-negative matrix factorization. Nature. 1999;401(6755):788–91. https://doi.org/10.1038/44565.

    Article  PubMed  CAS  Google Scholar 

  24. Babaee M, Tsoukalas S, Babaee M, Rigoll G, Datcu M. Discriminative nonnegative matrix factorization for dimensionality reduction. Neurocomputing. 2015;173(P2):212–23.

    Google Scholar 

  25. Linh NV, Anh NK, Than K. An effective NMF-based method for supervised dimension reduction. Advances in Intelligent Systems and Computing. 2015;326:93–104.

    Article  Google Scholar 

  26. Liu H, Wu Z, Cai D, et al. Constrained nonnegative matrix factorization for image representation. IEEE Transactions on Pattern Analysis & Machine Intelligence. 2012;34(7):1299–311. https://doi.org/10.1109/TPAMI.2011.217.

    Article  Google Scholar 

  27. Ding C, Li T, Jordan MI. Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis & Machine Intelligence. 2010;32(1):45–55. https://doi.org/10.1109/TPAMI.2008.277.

    Article  Google Scholar 

  28. Zdunek R. Improved convolutive and under-determined blind audio source separation with MRF smoothing. Cogn Comput. 2013;5(4):493–503. https://doi.org/10.1007/s12559-012-9185-9.

    Article  Google Scholar 

  29. Liu H, Sun F. Discovery of topical objects from video: a structured dictionary learning approach. Cogn Comput. 2016;8(3):519–28. https://doi.org/10.1007/s12559-015-9381-5.

    Article  Google Scholar 

  30. Xu XZ, Ding SF, Shi ZZ, et al. Optimizing radial basis function neural network based on rough sets and affinity propagation clustering algorithm. Frontiers of Information Technology & Electronic Engineering. 2012;13(2):131–8.

    CAS  Google Scholar 

  31. Frey BJ, Dueck D. Clustering by passing messages between data points. Science. 2007;315(5814):972–6. https://doi.org/10.1126/science.1136800.

    Article  PubMed  CAS  Google Scholar 

  32. Y. Lecun, C. Cortes, The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist/.

  33. The USPS handwritten digit data, http://www-i6.informatik.rwth-aachen.de/~keysers/usps.html

  34. Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006;70(1–3):489–501.

    Article  Google Scholar 

  35. Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans Inf Theory. 1967;13(1):21–7. https://doi.org/10.1109/TIT.1967.1053964.

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the Fundamental Research Funds for the Central Universities (No. 2015XKMS088)

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Dong Shan, Xinzheng Xu, Tianming Liang & Shifei Ding

Authors
  1. Dong Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinzheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianming Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinzheng Xu.

Ethics declarations

Conflict of Interest

Dong Shan, Xinzheng Xu, Tianming Liang, and Shifei Ding declare that they have no conflict of interest.

Informed Consent

Informed consent was not required as no human or animals were involved.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by the any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shan, D., Xu, X., Liang, T. et al. Rank-Adaptive Non-Negative Matrix Factorization. Cogn Comput 10, 506–515 (2018). https://doi.org/10.1007/s12559-018-9546-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-018-9546-0

Keywords

Search

Navigation