Manifold ranking graph regularization non-negative matrix factorization with global and local structures

Abstract

Non-negative matrix factorization (NMF) is a recently popularized technique for learning parts-based, linear representations of non-negative data. Although the decomposition rate of NMF is very fast, it still suffers from the following deficiency: It only revealed the local geometry structure; global geometric information of data set is ignored. This paper proposes a manifold ranking graph regularization non-negative matrix factorization with local and global geometric structure (MRLGNMF) to overcome the above deficiency. In particular, MRLGNMF induces manifold ranking to the non-negative matrix factorization with Sinkhorn distance. Numerical results show that the new algorithm is superior to the existing algorithm.

A Appendix

In order to prove Theorem 1 and deduce u, v update iterative formulas, similar to reference [2], we set Definition 1 [2] and Lemma 1 [2].

Definition 1

 [2] $G(\upsilon ,\upsilon ^{\prime })$ is an auxiliary function for F(v) if the conditions

$$\begin{aligned} G(\upsilon ,\upsilon ^{\prime })\ge F(v),\,\,G(v,v)=F(v), \end{aligned}$$

(A.1)

are satisfied.

The auxiliary function is very useful because of the following lemma.

Lemma 1

[2] IfGis an auxiliary function ofF, thenFis non-increasing under the update

$$\begin{aligned} \upsilon ^{t+1}=\arg \min _{v}G(v,v^{t}). \end{aligned}$$

(A.2)

Now, we will show that the update step for V in (3.4) with a proper auxiliary function. We rewrite the objective function ${\mathcal {O}}$ of MRLGNMF as follows:

$$\begin{aligned} {\mathcal {O}} &= d_{M}^{\lambda ,\gamma }(x,y)+{\mathcal {L}}_{F}+\gamma T_{r}(V^{T}LV)\nonumber \\ &= \min _{T_{pq}\ge 0}\sum _{p,q}(M\odot T)+\dfrac{1}{\lambda }H(T))+\gamma (\widetilde{KL}{(T1\Vert x)}\nonumber \\&\quad +\, \widetilde{KL}{(T^{T}1\Vert y)})\nonumber \\&\quad +\, \dfrac{\zeta }{2}\left( \sum ^{n}_{i,j=1}W_{ij}\left\| \frac{1}{\sqrt{D_{ii}}}F_{i}-\frac{1}{\sqrt{D_{jj}}}F_{j}\right\| ^{2}\right. \nonumber \\&\quad \left. +\mu \sum ^{n}_{i=1}\Vert F_{i}-V_{i}\Vert ^{2}\right) \nonumber \\&\quad +\, \sigma \sum ^{k}_{l=1}\sum ^{n}_{i=1}\sum ^{n}_{j=1}V_{jl}L_{ji}V_{il},\nonumber \\&\qquad \quad s.t.:U\ge 0,V\ge 0. \end{aligned}$$

(A.3)

Now, we prove the updated iteration formula u, v and Theorem 1.

Lemma 2

It can be seen from reference [2] that the function$G(v,v^{q})$is the auxiliary function of the objective function${\mathcal {O}}$. The function$G(v,v^{q})$is related to the auxiliary function ofv, where

$$\begin{aligned} G(\upsilon ,\upsilon ^{q}) &= G_{1}(v,v^{q})+G_{2}(v,v^{q})+G_{3}(v,v^{q}), \end{aligned}$$

(A.4)

$$\begin{aligned} G_{1}(v,v^{q}) &= \sum _{i}(x_{i}\log x_{i}-x_{i})+\sum _{ia}U_{ia}v_{a}\nonumber \\&-\, \sum _{ia}v_{i}\frac{U_{ia}v^{t}_{a}}{\sum _{b}U_{kb}v^{t}_{b}}\nonumber \\&\quad \left( \log U_{ia}v_{a}-\log \frac{U_{ia}v^{t}_{a}}{\sum _{b}U_{ib}v^{t}_{b}}\right) , \end{aligned}$$

(A.5)

$$\begin{aligned} G_{2}(v,v^{q}) &= z(v)+z^{\prime }(v)(v-v^{q})+\frac{1}{2}\frac{2\varphi (v+f)}{v}(v-v^{q})^{2}, \end{aligned}$$

(A.6)

$$\begin{aligned} G_{3}(v,v^{q}) &= \sigma \sum ^{k}_{l=1}\sum ^{n}_{i=1}\sum ^{n}_{j=1}V_{jl}L_{ji}V_{il}, \end{aligned}$$

(A.7)

Now, we set ${\mathcal {L}}_{\mathcal {O}}$ is Lagrange function of the objective function ${\mathcal {O}}$, and the ${\mathcal {L}}_{\mathcal {O}}$ is rewritten as

$$\begin{aligned} {\mathcal {L}}_{\mathcal {O}} &= \min _{T_{pq}\ge 0}\sum _{p,q}(M\odot T)+\dfrac{1}{\lambda }H(T))+\gamma (\widetilde{KL}{(T1\Vert x)}\nonumber \\&+\, \widetilde{KL}{(T^{T}1\Vert y)})\nonumber \\&+\, \dfrac{\zeta }{2}\left( \sum ^{n}_{i,j=1}W_{ij}\left\| \frac{1}{\sqrt{D_{ii}}}F_{i}-\frac{1}{\sqrt{D_{jj}}}F_{j}\right\| ^{2}\right. \nonumber \\&\left. +\, \mu \Sigma ^{n}_{i=1}\Vert F_{i}-V_{i}\Vert ^{2}\right) \nonumber \\&+\, \sigma \sum ^{k}_{l=1}\sum ^{n}_{i=1}\sum ^{n}_{j=1}V_{jl}L_{ji}V_{il}+Tr(\varphi U^{T})+Tr(\phi V^{T}). \end{aligned}$$

(A.8)

Then, we find partial derivatives of ${\mathcal {L}}_{\mathcal {O}}$ with respect to U and V, with KKT conditions $\varphi _{ij}u_{ij}=0$ and $\phi _{ij}v_{ij}=0$, and based on the above many parts, we get the objective function ${\mathcal {O}}$ for u and v for the update iteration formula as follows:

$$\begin{aligned}&u_{ik}\leftarrow u_{ik}\frac{\sum _{s}v_{sk}\frac{\sum _{t}T^{*it}_{s}}{y_{is}}}{\sum _{s}v_{sk}}, \end{aligned}$$

(A.9)

$$\begin{aligned}&v_{jk}\leftarrow v_{jk}\frac{\sum _{s}u_{sk}\frac{\sum _{t}T^{*st}_{j}}{y_{sj}}+\varphi F^{+}_{jk}+\sigma (WV)^{\frac{1}{2}}_{jk}}{\sum _{s}u_{sk}+\sigma (2DV)^{\frac{1}{2}}_{jk}+\varphi v_{jk}+\varphi F^{-}_{jk} }, \end{aligned}$$

(A.10)

where $\varphi =\zeta \mu$, F is updated in accordance with equation (3.2), and $F_{ij}^{+}=\frac{|F_{ij}|+F_{ij}}{2}$, $F_{ij}^{-}=\frac{|F_{ij}|-F_{ij}}{2}$.