Skip to main content
SpringerLink
Log in

An Efficient Parallelization Model for Sparse Non-negative Matrix Factorization Using cuSPARSE Library on Multi-GPU Platform

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13156))

Abstract

Positive or Non-negative Matrix Factorization (NMF) is an effective technique and has been widely used for Big Data representation. It aims to find two non-negative matrices W and H whose product provides an optimal approximation to the original input data matrix A, such that \(A\approx W*H\). Although, NMF plays an important role in several applications, such as machine learning, data analysis and biomedical applications. Due to the sparsity that is caused by missing information in many high-dimension scenes (e.g., social networks, recommender systems and DNA gene expressions), the NMF method cannot mine a more accurate representation from the explicit information. Therefore, the Sparse Non-negative Matrix Factorization (SNMF) can incorporate the intrinsic geometry of the data, which is combined with implicit information. Thus, SNMF can realize a more compact representation for the sparse data. In this paper, we study the Sparse Non-negative Matrix Factorization (SNMF). We use Multiplicative Update Algorithm (MUA) that computes the factorization by applying update on both matrices W and H. Accordingly, to address these issue, we propose a two models to implement a parallel version of SNMF on GPUs using NVIDIA CUDA framework. To optimize SNMF, we use cuSPARSE optimized library to compute the algebraic operations in MUA where sparse matrices A, W and H are stored in Compressed Sparse Row (CSR) format. At last, our contribution is validated through a series of experiments achieved on two input sets i.e. a set of randomly generated matrices and a set of benchmark matrices from real applications with different sizes and densities. We show that our algorithms allow performance improvements compared to baseline implementations. The speedup on multi-GPU platform can exceed \(11\times \) as well as the Ratio can exceed 91%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Tim Davis Matrix Collection. http://sparse.tamu.edu/

  2. Berry, M.W., Browne, M., Langville, A.N., Pauca, V.P., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Anal. 52(1), 155–173 (2007)

    Article  MathSciNet  Google Scholar 

  3. Bisot, V., Serizel, R., Essid, S., Richard, G.: Supervised nonnegative matrix factorization for acoustic scene classification. IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE), pp. 62–69 (2016)

    Google Scholar 

  4. Chen, X., Wu, K., Ding, M., Sang, N.: Sparse non-negative matrix factorizations for ultrasound factor analysis. Optik 124(23), 5891–5897 (2013)

    Article  Google Scholar 

  5. Comon, P., Jutten, C.: Handbook of Blind Source Separation: Independent component analysis and applications. Academic press (2010)

    Google Scholar 

  6. De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)

    Article  MathSciNet  Google Scholar 

  7. Devarajan, K.: Nonnegative matrix factorization: an analytical and interpretive tool in computational biology. PLoS Comput. Biol. 4(7) (2008)

    Google Scholar 

  8. Gao, Y., Church, G.: Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21(21), 3970–3975 (2005)

    Article  Google Scholar 

  9. Guo, Z., Zhang, Y.: A sparse corruption non-negative matrix factorization method and application in face image processing & recognition. Measurement 136, 429–437 (2019)

    Article  Google Scholar 

  10. He, P., Xu, X., Ding, J., Fan, B.: Low-rank nonnegative matrix factorization on stiefel manifold. Inf. Sci. 514, 131–148 (2020)

    Article  MathSciNet  Google Scholar 

  11. Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)

    Google Scholar 

  12. Huang, S., Wang, H., Li, T., Li, T., Xu, Z.: Robust graph regularized nonnegative matrix factorization for clustering. Data Min. Knowl. Disc. 32(2), 483–503 (2017). https://doi.org/10.1007/s10618-017-0543-9

    Article  MathSciNet  MATH  Google Scholar 

  13. Inuganti, S., Gampala, V.: Image compression using constrained non-negative matrix factorization. Int. J. 3(10) (2013)

    Google Scholar 

  14. Jia, Y.W.Y., Turk, C.H.M.: Fisher non-negative matrix factorization for learning local features. In: Proceedings of the Asian Conference on Computer Vision, pp. 27–30. Citeseer (2004)

    Google Scholar 

  15. Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philosophical Trans. Roy. Soc. A: Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)

    Article  MathSciNet  Google Scholar 

  16. Kannan, R., Ballard, G., Park, H.: A high-performance parallel algorithm for nonnegative matrix factorization. ACM SIGPLAN Notices 51(8), 1–11 (2016)

    Article  Google Scholar 

  17. Kannan, R., Woo, H., Aggarwal, C.C., Park, H.: Outlier detection for text data: An extended version. arXiv preprint arXiv:1701.01325 (2017)

  18. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)

    Article  Google Scholar 

  19. Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems, pp. 556–562 (2001)

    Google Scholar 

  20. Li, H., Li, K., Peng, J., Hu, J., Li, K.: An efficient parallelization approach for large-scale sparse non-negative matrix factorization using kullback-leibler divergence on multi-gpu. In: 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC), pp. 511–518. IEEE (2017)

    Google Scholar 

  21. Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized, parts-based representation. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, p. I. IEEE (2001)

    Google Scholar 

  22. Liao, J.C., Boscolo, R., Yang, Y.L., Tran, L.M., Sabatti, C., Roychowdhury, V.P.: Network component analysis: reconstruction of regulatory signals in biological systems. Proc. Natl. Acad. Sci. 100(26), 15522–15527 (2003)

    Article  Google Scholar 

  23. Liu, C., Yang, H.C., Fan, J., He, L.W., Wang, Y.M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on mapreduce. In: Proceedings of the 19th International Conference on World Wide Web, pp. 681–690 (2010)

    Google Scholar 

  24. Mejía-Roa, E., et al.: Biclustering and classification analysis in gene expression using nonnegative matrix factorization on multi-gpu systems. In: 2011 11th International Conference on Intelligent Systems Design and Applications, pp. 882–887. IEEE (2011)

    Google Scholar 

  25. Meng, Y., Shang, R., Jiao, L., Zhang, W., Yuan, Y., Yang, S.: Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering. Neurocomputing 290, 87–99 (2018)

    Article  Google Scholar 

  26. Moumni, H., Hamdi, O., Ezouaoui, S.: Algorithms and performance evaluation for sparse matrix product on grid’5000 intel xeon processor. In: 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), pp. 1–7. IEEE (2018)

    Google Scholar 

  27. Platoš, J., Gajdoš, P., Krömer, P., Snášel, V.: Non-negative matrix factorization on GPU. In: Zavoral, F., Yaghob, J., Pichappan, P., El-Qawasmeh, E. (eds.) NDT 2010. CCIS, vol. 87, pp. 21–30. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14292-5_4

    Chapter  Google Scholar 

  28. Tian, L.P., Luo, P., Wang, H., Zheng, H., Wu, F.X.: CASNMF: a converged algorithm for symmetrical nonnegative matrix factorization. Neurocomputing 275, 2031–2040 (2018)

    Article  Google Scholar 

  29. Trigeorgis, G., Bousmalis, K., Zafeiriou, S., Schuller, B.W.: A deep matrix factorization method for learning attribute representations. IEEE Trans. Pattern Anal. Mach. Intell. 39(3), 417–429 (2016)

    Article  Google Scholar 

  30. Vilamala, A., Lisboa, P.J., Ortega-Martorell, S., Vellido, A.: Discriminant convex non-negative matrix factorization for the classification of human brain tumours. Pattern Recogn. Lett. 34(14), 1734–1747 (2013)

    Article  Google Scholar 

  31. Wang, J.J.Y., Gao, X.: Max-min distance nonnegative matrix factorization. Neural Netw. 61, 75–84 (2015)

    Article  Google Scholar 

  32. Wang, S., Deng, C., Lin, W., Huang, G.B., Zhao, B.: NMF-based image quality assessment using extreme learning machine. IEEE Trans. Cybern. 47(1), 232–243 (2016)

    Article  Google Scholar 

  33. Yang, X., et al.: An integrated inverse space sparse representation framework for tumor classification. Pattern Recogn. 93, 293–311 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Sciences of Tunis, LIPSIC-FST Laboratory, University of Tunis El Manar, Manar II, 2092, Tunis, Tunisia

    Hatem Moumni & Olfa Hamdi-Larbi

  2. College of Business Madinah, Taibah University, Medina, Kingdom of Saudi Arabia

    Olfa Hamdi-Larbi

Authors
  1. Hatem Moumni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olfa Hamdi-Larbi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hatem Moumni .

Editor information

Editors and Affiliations

  1. Xiamen University, Xiamen, China

    Yongxuan Lai

  2. Beijing Normal University, Zhuhai, China

    Tian Wang

  3. Xiamen University, Xiamen, China

    Min Jiang

  4. Tianjin University, Tianjin, China

    Guangquan Xu

  5. Hunan University, Changsha, China

    Wei Liang

  6. University of Naples Parthenope, Naples, Italy

    Aniello Castiglione

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Moumni, H., Hamdi-Larbi, O. (2022). An Efficient Parallelization Model for Sparse Non-negative Matrix Factorization Using cuSPARSE Library on Multi-GPU Platform. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13156. Springer, Cham. https://doi.org/10.1007/978-3-030-95388-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95388-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95387-4

  • Online ISBN: 978-3-030-95388-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation