Abstract
Nonnegative matrix factorization (NMF) is a very attractive scheme in learning data representation, and constrained NMF further improves its ability. In this paper, we focus on the L2-norm constraint due to its wide applications in face recognition, hyperspectral unmixing, and so on. A new algorithm of NMF with fixed L2-norm constraint is proposed by using the Lagrange multiplier scheme. In our method, we derive the involved Lagrange multiplier and learning rate which are hard to tune. As a result, our method can preserve the constraint exactly during the iteration. Simulations in both computer-generated data and real-world data show the performance of our algorithm.







Similar content being viewed by others
References
AT&T Laboratories Cambridge, ORL Face Database. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html (2002)
M.W. Berry, M. Brown, A.N. Langvill, V.P. Pauca, R.J. Plemmons, Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Anal. 52(1), 155–173 (2007)
N. Bertin, R. Badeau, E. Vincent, Enforcing harmonicity and smoothness in Bayesian non-negative matrix factorization applied to polyphonic music transcription. IEEE Trans. Audio Speech Lang. Process. 18(3), 538–549 (2010)
C. Ding, T. Li, W. Peng, H. Park, Orthogonal nonnegative matrix t-factorizations for clustering, in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 126–135 (2006)
N. Gillis, F. Glineur, A multilevel approach for nonnegative matrix factorization. J. Comput. Appl. Math. 236(7), 1708–1723 (2012)
A. Gersho, R.M. Gray, Vector quantization and signal compression, in Springer International, vol. 159(1), pp. 407–485 (1992)
F. Gianfelici, C. Turchetti, P. Crippa, A non-probabilistic recognizer of stochastic signals based on KLT. Signal Process. 89(4), 422–437 (2009)
D. Goldfarb, Z. Wen, W. Yin, A curvilinear search method for the p-harmonic flow on spheres. SIAM J. Imaging Sci. 2(1), 84–109 (2009)
P.O. Hoyer, Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5(1), 1457–469 (2004)
I. Jolliffe, Principle Component Analysis, 2nd edn. (Springer, New York, 2004)
M. Lyons, J. Budynek, S. Akamatsu, Automatic classification of single facial images. IEEE Trans. Pattern Anal. Mach. Intell. 21(12), 1357–1362 (1999)
S. Li, X. Hou, H. Zhang, Q. Cheng, Learning spatially localized, parts-based representation, in Computer Vision and Pattern Recognition, pp. 207–212 (2001)
P. Luo, J. Peng, J. Fan, P. Luo, J. Peng, l(2,1) norm and Hessian regularized non-negative matrix factorization with discriminability for data representation. Appl. Sci. 7(10), 1–14 (2017)
D.D. Lee, H.S. Seung, Algorithms for nonnegative matrix factorization. Adv. Neural Inf. Process. Syst. 13(6), 556–562 (2000)
D.D. Lee, H.S. Seung, Learning the parts of objects by nonnegative matrix factorization. Nature 401(6755), 788–791 (1999)
Z. Li, J. Tang, X. He, Robust structured nonnegative matrix factorization for image representation. IEEE Trans. Neural Netw. Learn. Syst. 29(5), 1947–1960 (2018)
MIT Center For Biological and Computation Learning, CBCL Face Database #1. http://cbcl.mit.edu/software-datasets/FaceData2.html (2005)
Y. Meng, R. Shang, L. Jiao, W. Zhang, Y. Yuan, S. Yang, Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering. Neurocomputing 290, 87–99 (2018)
F. Shahnaz, M.W. Berry, V.P. Pauca, R.J. Plemmons, Document clustering using nonnegative matrix factorization. Inf. Process. Manag. 42(2), 373–386 (2006)
L.A. Vese, S.J. Osher, Numerical methods for p-harmonic flows and applications to image processing. SIAM J. Numer. Anal. 40(6), 2085–2104 (2003)
D. Wang, H. Lu, On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Process. 93(6), 1608–1623 (2013)
D. Wang, J. Liu, Y. Gao, J. Yu, C. Zheng, An NMF-L2, L1-norm constraint method for characteristic gene selection. Plos One 11(7), 1–12 (2016)
Z. Wen, W. Yin, A feasible method for optimization with orthogonality constraints. Math. Program. 142(1–2), 394–434 (2013)
Y. Xu, Z. Zhong, J. Yang, J. You, D. Zhang, A new discriminative sparse representation method for robust face recognition via l2 regularization. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2233–2242 (2017)
N. Yuvaraj, P. Vivekanandan, An efficient SVM based tumor classification with symmetry non-negative matrix factorization using gene expression data, in International Conference on Information Communication and Embedded Systems, pp. 761–768 (2013)
Z. Yang, Y. Xiang, K. Xie, Y. Lai, Adaptive method for nonsmooth nonnegative matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 28(4), 948–960 (2017)
Z. Yang, G. Zhou, S. Xie, S. Ding, J. Yang, J. Zhang, Blind spectral unmixing based on sparse nonnegative matrix factorization. IEEE Trans. Image Process. 20(4), 1112–1125 (2011)
Z. Yang, Y. Zhang, Y. Xiang, W. Yan, S. Xie, Non-negative matrix factorization with dual constraints for image clustering. IEEE Trans. Syst. Man Cybern. Syst. https://doi.org/10.1109/TSMC.2018.2820084 (to appear)
Z. Zhang, S. Liao, H. Zhang, S. Wang, Y. Wang, Bilateral filter regularized L2 sparse nonnegative matrix factorization for hyperspectral unmixing. Remote Sens. 10(6), 1–22 (2018)
G. Zhou, Z. Yang, S. Xie, J. Yang, Online blind source separation using incremental nonnegative matrix factorization with volume constraint. IEEE Trans. Neural Netw. 22(4), 550–60 (2011)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by the National Natural Science Foundation of China under Grant 61722304, the Pearl River S&T Nova Program of Guangzhou under Grant 201610010196, and the Guangdong Natural Science Funds under Grants 2014A030306037, 2018A030313306.
Appendix
Appendix
1.1 The Proof of Lemma 2
Proof
Part (1): Since \(\mathbf{I }+\frac{\tau }{2}\mathbf{A }=\mathbf{I }+\frac{\tau }{2}\mathbf{MN }^\mathrm{T}\), we apply the SMW formula:
In the case that \(\mathbf{B } = \mathbf{I }\), it obtains that \((\mathbf{I }+\frac{\tau }{2}\mathbf{A })^{-1}=\mathbf{I }-\frac{\tau }{2}\mathbf{M }(\mathbf{I }+\frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M })^{-1}\mathbf{N }^\mathrm{T}\). As \(\mathbf{I }-\frac{\tau }{2}\mathbf{A }=\mathbf{I }-\frac{\tau }{2}\mathbf{MN }^\mathrm{T}\), we have
Part (2): When \(\mathbf{A }=\mathbf{a }\mathbf{x }^\mathrm{T}-\mathbf{x }\mathbf{a }^\mathrm{T}\), then \(\mathbf{M }=[\mathbf{a },\mathbf{x }]\) and \(\mathbf{N }=[{\mathbf{a }},-\mathbf{a }]\), let \(\mathbf{S }=\mathbf{I }+\frac{\tau }{2}\mathbf{N }^\mathrm{T}\mathbf{M }\),
Since \(\mathbf{S }^{-1}=\frac{\mathbf{S }^{*}}{|\mathbf{S }|}\), and
Let \(k=1-(\frac{\tau }{2})^{2}(\mathbf{a }^\mathrm{T}\mathbf{x })^{2}+ (\frac{\tau }{2})^{2}\parallel \mathbf{a }\parallel _{2}^{2}\parallel \mathbf{x }\parallel _{2}^{2}\), then
Rights and permissions
About this article
Cite this article
Yang, Z., Hu, Y., Liang, N. et al. Nonnegative Matrix Factorization with Fixed L2-Norm Constraint. Circuits Syst Signal Process 38, 3211–3226 (2019). https://doi.org/10.1007/s00034-018-1012-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00034-018-1012-4