Abstract
Nonnegative matrix factorization (NMF) is a popular matrix decomposition technique that has proven to be useful across a diverse variety of fields. Over the years, several algorithms have been proposed to improve the convergence of iterative algorithms in NMF, such as the multiplicative, the projected gradient, the second-order algorithms and recently the projected conjugate gradient algorithms. However, most of these procedures suffer from either slow convergence or numerical instability. In this paper, we propose a projected hybrid conjugate gradient algorithm which avoids the slow convergence problem by using orthogonal searching direction at each step, which ensures that is a descent direction, also avoids jamming that occurs in other conjugate gradient methods such as Fletcher–Reeves. We presented experimental results on both synthetic and real-world datasets that demonstrate the superiority of our algorithm, both in terms of better approximations as well as computational efficiency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Lee, D., Seung, H.: Algorithms for non-negative matrix factorization. Adv. Neural Inf. Process. Syst. 13, 556–562 (2001)
Sajda, P., Du, S., Brown, T., Stoyanova, R., Shungu, D., Mao, X., et al.: Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain. IEEE Trans. Med. Imaging 23(12), 1453–1465 (2004)
Shahnaz, F., Berry, M.: Document clustering using nonnegative matrix factorization. Inf. Process. Manag. 42, 373–386 (2006)
Xu, W., Liu, X., Gong, Y.: Document-clustering based on non-negative matrix factorization. In: Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, vol. 28, pp 267–273 (2003)
Guillamet, D., Vitri a, J.: Non-negative matrix factorization for face recognition. Top. Artif. Intell. 2504, 336–344 (2002)
Li, S., Hou, X., Zhang, H., Cheng, Q.: Learning spatially localized, parts-based representation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 207–212 (2001)
Ziehe, A., Laskov, P., Pawelzik, K., Mueller, K.: Non-negative sparse coding for general blind source separation. In: Advance Neural Information Processing Systems, vol. 16, Vancouver, Canada (2003)
Chu, M., Diele, F., Plemmons, R., Ragni, S.: Optimality, computation, and interpretation of nonnegative matrix factorizations. http://www.wfu.edu/~plemmons (2004)
Smaragdis, P., Brown, J.: Non-negative matrix factorization for polyphonic music transcription. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 177–180 (2004)
Lee, D., Seung, H.: Learning the parts of objects by non-negative matrix factorization. Nature. 401(6755), 788–791 (1999)
Khedr, W.M., Abd El-Aziz, M.E., Amer, S.M.: A Novel Algorithm for multichannel deconvolutive based on \({\alpha \beta }\)-divergence. Int. J. Comput. Appl. 57(10), 9–14 (2012)
Ozerov, A., Fevotte, C.: Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation. IEEE Trans. Audio, Speech Lang. Process. 18(3), 550–563 (2010)
Lin”, C.: On the convergence of multiplicative update algorithms for nonnegative matrix factorization. IEEE Trans. Neural Netw. 18(6), 1589–1596 (2007)
Kim, D., Sra, S., Dhillon, I.: Fast projection-based methods for the least squares nonnegative matrix approximation problem. Stat. Anal. Data Min. 1(1), 38–51 (2008)
Cichocki, A., Zdunek, R.: Regularized alternating least squares algorithms for nonnegative matrix/tensor factorizations. In: Proceedings of the 4th International Symposium on Neural Networks, Nanjing, China, June 9, 2007, pp. 793–802 (2007)
Lin, C.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)
Cichocki, A., Zdunek, R.: Multilayer nonnegative matrix factorization using projected gradient approaches. Int. J. Neural Syst. 17(6), 431–446 (2007)
Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with constrained second-order optimization. Signal Process. 87(8), 1904–1916 (2007)
Wang, W., Zou, X.: Non-Negative matrix factorization based on projected nonlinear conjugate gradient algorithm. In: Proceedings of the ICA Research Network International Workshop (ICARN 2008), Liverpool, UK, 25–26 September 2008, pp. 5–8 (2008)
Andrei, N.: hybrid conjugate gradient algorithm for unconstrained optimization. J. Optim. Theory Appl. 14(1), 249–264 (2009)
Kafaki, S., Amiri, N.: Two modified hybrid conjugate gradient methods based on a hybrid secant equation. Math. Model. Anal. 18(1), 32–52 (2013)
Li, Xiangrong, Zhao”, Xupei: A hybrid conjugate gradient method for optimization problems. Nat. Sci. 3(1), 85–90 (2011)
Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Technical Report, Carnegie Mellon University (1994)
ORL Database of Faces. AT&T Laboratories, Cambridge. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html (1999)
CBCL Face Database. Center for Biological and Computational Learning at MIT and MIT. http://cbcl.mit.edu/software-datasets/FaceData2.html (2000)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Abd El Aziz, M., Khidr, W. Nonnegative matrix factorization based on projected hybrid conjugate gradient algorithm. SIViP 9, 1825–1831 (2015). https://doi.org/10.1007/s11760-014-0661-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-014-0661-4