Correntropy induced metric based graph regularized non-negative matrix factorization
Introduction
Dimension reduction makes an important contribution in pattern recognition, subspace selection [23], computer vision [3], [7], [9], [13] and information retrieval [6], [5], [8]. Since dimension reduction reveals the intrinsic structure of data, it can enhance the performance of consequent processing. Among existing dimension reduction methods, non-negative matrix factorization (NMF) [4], [10], [11] has received much attention and become a hot topic in recent years. Particularly, NMF learns two low-dimensional matrices to approximate the original high-dimensional data matrix, meanwhile constrains them to be non-negative. Since NMF can learn a natural parts-based representation, which is consistent with the intuition of learning the parts to form a whole, it has been widely applied to data mining [12], [14], [15], [17], pattern recognition [18], [19], [20], [21], and computer vision [16], [22], [24], [25].
Since the seminal work of Lee and Seung [4], NMF has been continuously improved. For example, Zeferiou et al. [26] proposed discriminant NMF (DNMF) for supervised dimension reduction by incorporating Fisher׳s criterion in NMF. However, DNMF requires the samples obey Gaussian distribution, which is sometimes inconsistent with the assumption of NMF. Cai et al. [27] developed a graph regularized NMF (GNMF) to encode the geometric structure of data by a nearest neighbor (NN) graph. However, traditional NMF methods cannot provide a robust decomposition because their objective function, i.e., L2-norm based [28] and Kullback–Leibler (KL) divergence based [29] loss functions, are sensitive to outliers. Although Li et al. [33] proposed a robust graph regularized NMF via maximizing correntropy criterion (MCCGR), the constructed adjacent graph is influenced by noisy samples, and thus leads to poor performance on seriously corrupted datasets. From the viewpoint of learning, both robustness of data representation and purity of constructed graph are important in NMF.
In this paper, we propose a correntropy induced metric based graph regularized NMF (CGNMF) to improve the robustness of NMF with the geometric structure of dataset preserved. In particular, we replace the L2-norm based loss of GNMF with the well-known correntropy induced metric (CIM, [30]) to search a robust matrix decomposition. Since CIM approximates the L0-norm when the volume of error is large and approximates the L2-norm when the volume of error is relatively small, it is robust to noise of large magnitudes. CGNMF learns a low-dimensional subspace which preserves the intrinsic geometric structure of dataset via a previously constructed graph from the original data. In addition, to improve the purity of the constructed graph, we improved our CGNMF by constructing the graph with the sparse representation method [31], [32]. To enhance the reliability of CGNMF, we proposed correntropy induced metric based graph regularized projective NMF (CGPNMF) to learn clean coefficients by narrowing its distance to the projected samples measured in the correntropy induced metric sense. Experimental results on popular facial image datasets confirm the effectiveness of both CGNMF and CGPNMF comparing with the state-of-the-arts methods.
The rest of this paper is organized as follows: we briefly reviewed the related NMF variants in Section 2 and presented the correntropy induced metric based graph regularized NMF (CGNMF), its stable version, and the optimization algorithm in Section 3. Section 4 proposed the correntropy induced metric based graph regularized projective NMF (CGPNMF) and its optimization algorithms as well as the proof of convergence. Then we show the experimental results on popular facial image datasets comparing with the representative NMF methods in Section 5. We conclude this paper in Section 6.
Section snippets
Related works
NMF aims to find two non-negative matrices, i.e., and , to approximate the sample data, i.e., , by minimizing the distance between X and UV, where . Traditional measurement is either squared L2-norm or Kullback–Leibler (KL)-divergence, and they are not robust enough because their underlying distributions cannot effectively model outliers.
To well preserve the intrinsic geometric structure of the original data, Cai et al. [27] proposed graph regularized NMF (GNNF)
Correntropy induced metric based graph regularized NMF
In this paper, we first presented a correntropy induced metric based graph regularized NMF (CGNMF) based on the correntropy induced metric [30]. Secondly, we improved the reliability of CGNMF by constructing the adjacent graph with sparse representation.
Correntropy induced metric based graph regularized projective NMF
Although CGNMF can learn an effective low dimensional space by inhibiting the influence of outliers to both coefficients and adjacent graph, it is unsatisfactory on some seriously corrupted datasets because the CIM based loss still introduces noisy on the learned coefficients. In this paper, we proposed a correntropy induced metric based graph regularized projective NMF (CGPNMF) by directly inhibiting the effect of outliers to the coefficients. In particular, based on CGNMF, CGPNMF further
Experimental results
In this section, we present several experiments to evaluate the effectiveness of the proposed CGNMF and CGPNMF on both Yale [39] and ORL datasets [40], comparing with GNMF [27], MCCGR [33], RMNMF [34], K-means [41], [42], NMF [28]. Since the initial U and V are selected randomly, we performed 20 independent trials with different number of clusters to compute the average accuracy and the average mutual information. We initialized all algorithms with the same randomly generated U and V, and then
Conclusion
In this paper, we first proposed a correntropy induced metric based graph regularized NMF (CGNMF), and a stable version with sparse representation based adjacent graph construction, and then proposed a correntropy induced metric based graph regularized projective NMF (CGPNMF) by learning clean coefficients. Since CGNMF maximizes the correntropy to denoise the data and constrains the coefficient representation with adjacent graph learned from the original data, it outperforms the other
Acknowledgments
This work was partially supported by the Research Fund for the Doctoral Program of Higher Education of China, SRFDP (under Grant no. 20134307110017) and the Scientific Research Plan Project of NUDT (under Grant no. JC13-06-01 and JC14-06-01) and the National Natural Science Foundation of China (under Grant no. 61502515).
Yuanyuan Wang received both B.S. and M.S. degrees from the National University of Defense Technology, Changsha, China. Now she is a lecturer with the Army Officer Academy. His current research interests include computer vision and machine learning.
References (42)
- et al.
Multiview Hessian discriminative sparse coding for image annotation
Comput. Vis. Image Underst.
(2014) - et al.
Learning dictionary on manifolds for image classification
Pattern Recognit.
(2013) - B. Mao, N.Y. Guan, D.C. Tao, X.H. Huang, Z.G. Luo, Correntropy induced metric based graph regularized non-negative...
- W.J. Zhang, N.Y. Guan, D.C. Tao, B. Mao, X.H. Huang, Z.G. Luo, Correntropy supervised non-negative matrix...
- et al.
General tensor discriminant analysis and Gabor features for gait recognition
IEEE Trans. Pattern Anal. Mach. Intell.
(2007) - et al.
Learning the parts of objects by non-negative matrix factorization
Nature
(1999) - et al.
Learning to rank using user clicks and visual features for image retrieval
IEEE Trans. Cybern.
(2015) - et al.
Asymmetric bagging and random subspace for support vector machines based relevance feedback in image retrieval
IEEE Trans. Pattern Anal. Mach. Intell.
(2006) - et al.
Multiview intact space learning
IEEE Trans. Pattern Anal. Mach. Intell.
(2015) - et al.
Click prediction for web image reranking using multimodal sparse coding
IEEE Trans. Image Process.
(2014)
High-order distance based multiview stochastic learning in image classification
IEEE Trans. Cybern.
Nonnegative patch alignment framework
IEEE Trans. Neural Netw.
Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent
IEEE Trans. Image Process.
Adaptive hypergraph learning and its applications in image classification
IEEE Trans. Image Process.
Ensemble manifold regularization
IEEE Trans. Pattern Anal. Mach. Intell.
NeNMFAn optimal gradient method for nonnegative matrix factorization
IEEE Trans. Signal Process.
Online nonnegative matrix factorization with robust stochastic approximation
IEEE Trans. Neural Netw. Learn. Syst.
Nonsmooth nonnegative matrix factorization (nsNMF)
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (21)
Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints
2024, Engineering Applications of Artificial IntelligenceNoise-tolerant clustering via joint doubly stochastic matrix regularization and dual sparse coding
2023, Expert Systems with ApplicationsNoise-aware clustering based on maximum correntropy criterion and adaptive graph regularization
2023, Information SciencesRobust semi-supervised nonnegative matrix factorization for image clustering
2021, Pattern RecognitionCitation Excerpt :For example, the graph regularized NMF methods have been proposed by using a nearest neighborhood graph to explore the intrinsic geometrical structure of data [17–20]. In order to suppress the negative influence of non-Gaussian noise with complicated statistical distribution in the data, several robust loss functions such as l2,p-norm and correntropy have been applied in NMF for developing some robust NMF methods [21–24]. Up to now, most of the existing NMF approaches are unsupervised, and pay little attention to the supervised information hidden in the data.
Robust non-negative matrix factorization with multiple correntropy-induced hypergraph regularizer
2020, Signal ProcessingCitation Excerpt :Our method is closely relevant to correntropy induced NMF. Wang et al. [14] Proposed a Correntropy induced graph egularized NMF method, where the correntropy induced metric is used to measure the reconstruction errors and the simple graph Laplacian regularizer is used to preserve the intrinsic manifold structure. Zhang et al. [15] proposed Correntropy Supervised NMF, where not only the correntropy metrix is used, but also the data label information is leveraged to employ the classification task.
Robust orthogonal nonnegative matrix tri-factorization for data representation
2020, Knowledge-Based Systems
Yuanyuan Wang received both B.S. and M.S. degrees from the National University of Defense Technology, Changsha, China. Now she is a lecturer with the Army Officer Academy. His current research interests include computer vision and machine learning.
Shuyi Wu received the B.S. degree from the National University of Defense Technology, Changsha, China, where he is currently working toward the M.S. degree with the School of Computer Science. His current research interests include computer vision and image processing.
Bin Mao received the both B.S. and M.S. degrees from the National University of Defense Technology, Changsha, China, where he is currently working toward the Ph.D. degree with the School of Computer Science. His current research interests include machine learning and data mining.
Xiang Zhang received the B.S. and M.S. degrees from the Anhui University and National University of Defense Technology, respectively. He is currently working toward his Ph.D. degree with the School of Computer Science. His current research interests include computer vision and image processing.
Zhigang Luo received the B.S., M.S., and Ph.D. degrees from the National University of Defense Technology, Changsha, China, where he is currently a Professor with the School of Computer Science. His current research interests include artificial intelligence and machine learning.