Abstract
Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.
Similar content being viewed by others
References
Battenberg, E., Wessel, D.: Accelerating non-negative matrix factorization for audio source separation on multi-core and many-core architectures. In: International Society for Music Information Retrieval Conference, Ismir 2009, pp. 501–506. Kobe International Conference Center, Kobe, Japan (2009)
Bharti, S.K., Babu, K.S.: Automatic keyword extraction for text summarization: a survey. CoRR abs/1704.03242. http://arxiv.org/abs/1704.03242 (2017)
Boutsidis, C., Gallopoulos, E.: Svd based initialization: a head start for nonnegative matrix factorization. Pattern Recogn. 41(4), 1350–1362 (2008)
Chang, C.H., Kehtarnavaz, N.: Computationally efficient image deblurring using low rank image approximation and its gpu implementation. J. Real-Time Image Proc. 12(3), 567–573 (2016)
Chen, W.S., Li, Y., Pan, B., Xu, C.: Fast non-negative matrix factorizations for face recognition. Int. J. Pattern Recog. Artificial Intell. 32(04):1856,003 (2018)
Cheng, J., Grossman, M., McKercher, T.: Professional cuda C programming. Wiley, New York (2014)
Georgis G., Lentaris, G., Reisis, D.: Acceleration techniques and evaluation on multi-core cpu, gpu and fpga for image processing and super-resolution. J. Real-Time Image Process. https://doi.org/10.1007/s11554-016-0619-6 (2016)
Gillis, N.: Introduction to nonnegative matrix factorization. CoRR abs/1703.00663. http://arxiv.org/abs/1703.00663 (2017)
Huang, K., Sidiropoulos, N.D., Swami, A.: Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition. IEEE Trans. Signal Process. 62(1), 211–224 (2014)
Jošth, R., Antikainen, J., Havel, J., Herout, A., Zemčík, P., Hauta-Kasari, M.: Real-time pca calculation for spectral imaging (using simd and gp-gpu). J. Real-Time Image Proc. 7(2), 95–103 (2012)
Kalman, D.: A singularly valuable decomposition: the svd of a matrix. Coll. Math. J. 27(1), 2–23 (1996)
Kang, M., Kim, J.M.: Singular value decomposition based feature extraction approaches for classifying faults of induction motors. Mech. Syst. Signal Process. 41(1–2), 348–356 (2013)
Kannan, R., Ballard, G., Park, H.: A high-performance parallel algorithm for nonnegative matrix factorization. CoRR. http://arxiv.org/abs/1509.09313 (2016)
Kong”, W.: Technique for image fusion based on NSST domain INMF. Optik Int. J. Light Electron Opt. 125(11), 2716–2722 (2014)
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Leen, T. K., Dietterich, T. G., Tresp, V. (eds). Advances in Neural Information Processing Systems 13, vol. 13, pp. 556–562. Massachusetts, MIT Press (2001)
Lijie, C.: Singular value decomposition applied to digital image processing. Division of Computing Studies, Arizona State University Polytechnic Campus, Mesa, Arizona State University polytechnic Campus, pp. 1150–1165 (2006)
Mejía-Roa, E., Tabas-Madrid, D., Setoain, J., García, C., Tirado, F., Pascual-Montano, A.: Nmf-mgpu: non-negative matrix factorization on multi-gpu systems. BMC Bioinform. 16(1), 43 (2015)
Pacheco, P.: An Introduction to Parallel Programming, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2011)
Platoš J., Gajdoš P., Krömer P., Snášel V.: Non-negative Matrix Factorization on GPU. In: Zavoral F., Yaghob J., Pichappan, P., El-Qawasmeh, E. (eds.) Networked Digital Technologies, vol. 87, pp. 21–30. Springer, Berlin, Heidelberg (2010)
Qiao, H.: New svd based initialization strategy for non-negative matrix factorization. Pattern Recogn. Lett. 63, 71–77 (2015)
Shlens, J.: A tutorial on principal component analysis. CoRR abs/1404.1100. (2014). http://arxiv.org/abs/1404.1100
Soniwal, Y., Mitra, A., Venkatesh, K.S.: Entire frame image display employing monotonic convergent nonnegative matrix factorization. J. Real-Time Image Process. (2017). https://doi.org/10.1007/s11554-017-0730-3. https://doi.org/10.1007/s11554-017-0730-3
Sorzano, C.O.S., Vargas, J., Montano, A.P.: A survey of dimensionality reduction techniques. arXiv preprint arXiv:14032877 (2014)
Tao, D., Tao, D., Li, X., Gao, X.: Large sparse cone non-negative matrix factorization for image annotation. ACM Trans. Intell. Syst. Technol. (TIST) 8(3), 37 (2017)
Wang, Y.X., Zhang, Y.J.: Nonnegative matrix factorization: A comprehensive review. IEEE Trans. Knowl. Data Eng. 25(6), 1336–1353 (2013)
Xu, Y., Deng, S., Li, X., He, Y.: A sparse unmixing model based on nmf and its application in raman image. Neurocomputing 207(C), 120–130 (2016)
Yanez, F., Bach, F.R.: Primal-dual algorithms for non-negative matrix factorization with the kullback-leibler divergence. CoRR. http://arxiv.org/abs/1412.1788 (2014)
Yong, W., Yuqing, W., Xiaohui, Z.: Complex number-based image quality assessment using singular value decomposition. IET Image Proc. 10(2), 113–120 (2016)
Zekri, H., Mokhtari, A.R., Cohen, D.R.: Application of singular value decomposition (svd) and semi-discrete decomposition (sdd) techniques in clustering of geochemical data: an environmental study in central iran. Stoch. Env. Res. Risk Assess. 30(7), 1947–1960 (2016)
Acknowledgements
The authors would like to thank the associate editor and anonymous reviewers for their comments, which significantly helped improve this paper. This work is supported by the Research Fund of State Key Laboratory of High-Performance Computing, under Grant No. 201612-01.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Song, M., Peng, Y., Jiang, T. et al. Accelerated image factorization based on improved NMF algorithm. J Real-Time Image Proc 15, 93–105 (2018). https://doi.org/10.1007/s11554-018-0785-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-018-0785-9