Enhancing social collaborative filtering through the application of non-negative matrix factorization and exponential random graph models

Abstract

Social collaborative filtering recommender systems extend the traditional user-to-item interaction with explicit user-to-user relationships, thereby allowing for a wider exploration of correlations among users and items, that potentially lead to better recommendations. A number of methods have been proposed in the direction of exploring the social network, either locally (i.e. the vicinity of each user) or globally. In this paper, we propose a novel methodology for collaborative filtering social recommendation that tries to combine the merits of both the aforementioned approaches, based on the soft-clustering of the Friend-of-a-Friend (FoaF) network of each user. This task is accomplished by the non-negative factorization of the adjacency matrix of the FoaF graph, while the edge-centric logic of the factorization algorithm is ameliorated by incorporating more general structural properties of the graph, such as the number of edges and stars, through the introduction of the exponential random graph models. The preliminary results obtained reveal the potential of this idea.

Appendices

Appendix 1: Approximating the free energy of the 2-star model

The analysis that follows aims to show how the exact form of the free energy of the 2-star model is derived. Starting with the Hamiltonian of the aforementioned model (Eqs. 16 and 32 below)

$$\begin{aligned} H = \frac{J}{n-1}\sum \limits _{i=1}^{n} (k_{i})^2 + B\sum \limits _{i=1}^{n} k_{i} \end{aligned}$$

(32)

the partition function Z becomes

$$\begin{aligned} Z= & {} \sum \limits _{G \in \mathcal {G}} \mathrm {e}^{H(G)} = \sum \limits _{G \in \mathcal {G}} \exp \left\{ \frac{J}{n-1}\sum \limits _{i=1}^{n} (k_{i})^2 + B\sum \limits _{i=1}^{n} k_{i}\right\} \nonumber \\= & {} \sum \limits _{G \in \mathcal {G}} \exp \left\{ \frac{J}{n-1}\sum \limits _{i=1}^{n} (k_{i})^2 \right\} \times \exp \left\{ B\sum \limits _{i=1}^{n} k_{i}\right\} \end{aligned}$$

(33)

According to Park and Newman (2004b), sums like that of Eq. 33 (containing terms of the form of $\mathrm {e}^{k^2}$) are encountered in the study of interacting quantum systems and may be calculated through the application of the Hubbard–Stratonovich transformation (Hubbard 1959). The said transformation is used for the conversion of a particle theory (in this case, the node degree k) to the corresponding field theory, through the introduction of an auxiliary scalar field (in this case, $\phi _i$, as it will be demonstrated next).

The Hubbard-Stratonovich transformation is based upon the Gaussian Integrals, that may be written in any of the following two forms

$$\begin{aligned}&\int _{-\infty }^{\infty } \mathrm {e}^{-\alpha x^2} \mathrm {d}x = \sqrt{\frac{\pi }{\alpha }} \end{aligned}$$

(34)

$$\begin{aligned}&\int _{-\infty }^{\infty }\mathrm {e}^{- a x^2 + b x + c}\,dx = \sqrt{\frac{\pi }{a}}\,\mathrm {e}^{\frac{b^2}{4a}+c} \end{aligned}$$

(35)

with the real constant $\alpha$ taking non-negative values ($\alpha > 0$). Substituting $\alpha$ and x according to Eq. 36 below

$$\begin{aligned} \alpha \leftarrow (n-1)J, \quad x \leftarrow \phi _i - \frac{k_i}{n-1} \end{aligned}$$

(36)

and subsequently plugging them into the Gaussian Integral of Eq. 34, yields

$$\begin{aligned}&\int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\left( \phi _i - \frac{k_i}{n-1}\right) ^2 \right\} \mathrm {d}\left( \phi _i - \frac{k_i}{n-1}\right) = \sqrt{\frac{\pi }{(n-1)J}} \Rightarrow \nonumber \\&\int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i - \frac{Jk_i^2}{n-1}\right\} \mathrm {d}\left( \phi _i - \frac{k_i}{n-1}\right) = \sqrt{\frac{\pi }{(n-1)J}} \Rightarrow \nonumber \\&\mathrm {e}^{-\frac{Jk_i^2}{n-1}}\times \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i\right\} \mathrm {d}\phi _i -\nonumber \\&\qquad - \frac{1}{n-1} \mathrm {e}^{-(n-1)J\phi _i^2} \times \int _{-\infty }^{\infty } \exp \left\{ 2Jk_i\phi _i - \frac{Jk_i^2}{n-1}\right\} \mathrm {d}k_i = \sqrt{\frac{\pi }{(n-1)J}} \end{aligned}$$

(37)

In order to eliminate the integral in Eq. 37, the second form of the Gaussian Integral is being used (Eq. 35). Substituting the constants $\alpha , b, c$ and the unknown variable x according to Eq. 38 below

$$\begin{aligned} a \leftarrow \frac{J}{n-1}, \quad b \leftarrow 2J\phi _i, \quad c \leftarrow 0, \quad x \leftarrow k_i \end{aligned}$$

(38)

yields

$$\begin{aligned} \int _{-\infty }^{\infty } \exp \left\{ 2Jk_i\phi _i - \frac{Jk_i^2}{n-1}\right\} \mathrm {d}k_i = \sqrt{\frac{(n-1)\pi }{J}}\mathrm {e}^{(n-1)J\phi _i^2} \end{aligned}$$

(39)

The second term of the left-hand side of Eq. 37 in conjunction with Eq. 39, becomes

$$\begin{aligned} - \frac{1}{n-1} \mathrm {e}^{-(n-1)J\phi _i^2}\sqrt{\frac{(n-1)\pi }{J}}\mathrm {e}^{(n-1)J\phi _i^2}=-\sqrt{\frac{\pi }{(n-1)J}} \end{aligned}$$

(40)

and Eq. 37 in conjunction with Eq. 40 becomes

$$\begin{aligned}&\mathrm {e}^{-\frac{Jk_i^2}{n-1}}\times \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i\right\} \mathrm {d}\phi _i -\sqrt{\frac{\pi }{(n-1)J}} = \sqrt{\frac{\pi }{(n-1)J}} \Rightarrow \nonumber \\&\mathrm {e}^{-\frac{Jk_i^2}{n-1}}\times \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i\right\} \mathrm {d}\phi _i = 2\sqrt{\frac{\pi }{(n-1)J}} \Rightarrow \nonumber \\&\mathrm {e}^{\frac{J}{n-1}k_i^2} =\frac{1}{2}\sqrt{\frac{(n-1)J}{\pi }}\times \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i\right\} \mathrm {d}\phi _i \end{aligned}$$

(41)

Taking the product of the left-hand side of Eq. 41 for all n nodes of the graph and making use of the property of iterated integrals for multiple-variable functions, yields

$$\begin{aligned}&\exp \left\{ \frac{J}{n-1} \sum \limits _{i=1}^{n} (k_i)^2 \right\} = \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \times \prod _{i=1}^{n} \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\phi _i^2 + 2Jk_i\phi _i\right\} \nonumber \\&\qquad \mathrm {d}\phi _i \Rightarrow \nonumber \\&\exp \left\{ \frac{J}{n-1} \sum \limits _{i=1}^{n} (k_i)^2 \right\} = \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \times \nonumber \\&\qquad \times \int _{-\infty }^{\infty } \ldots \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + 2J\sum \limits _{i=1}^{n}k_i\phi _i \right\} \mathrm {d}\phi _{1}\ldots \mathrm {d}\phi _{n} \end{aligned}$$

(42)

The integral of the right-hand side of Eq. 42 is once again calculated based on quantum mechanics (Park and Newman 2004a). More specifically, every $\phi _i$ is thought to symbolize the contribution of a respective field during the movement of a particle in an one-dimensional system. Consequently, the effect of the overall field in the particle movement is approximated by the superposition of each individual field $\phi _i$, which is mathematically formulated as a path integral (Eq. 43)

$$\begin{aligned}&\int _{-\infty }^{\infty } \ldots \int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + 2J\sum \limits _{i=1}^{n}k_i\phi _i \right\} \mathrm {d}\phi _{1}\ldots \mathrm {d}\phi _{n} \nonumber \\&\qquad =\int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + 2J\sum \limits _{i=1}^{n}k_i\phi _i \right\} \mathscr {D}\phi \end{aligned}$$

(43)

As a consequence, Eq. 42 is rewritten in the form

$$\begin{aligned}&\exp \left\{ \frac{J}{n-1} \sum \limits _{i=1}^{n} (k_i)^2 \right\} = \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}}\int _{-\infty }^{\infty }\nonumber \\&\exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + 2J\sum \limits _{i=1}^{n}k_i\phi _i \right\} \mathscr {D}\phi \end{aligned}$$

(44)

Substituting Eq. 44 into Eq. 33, the partition function Z becomes

$$\begin{aligned} Z= & {} \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \sum \limits _{G \in \mathcal {G}}\int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + 2J\sum \limits _{i=1}^{n}k_i\phi _i \right\} \mathscr {D}\phi \nonumber \\&\times \exp \left\{ B\sum \limits _{i=1}^{n} k_{i} \right\} \Rightarrow \nonumber \\ Z= & {} \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \sum \limits _{G \in \mathcal {G}}\int _{-\infty }^{\infty } \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \sum \limits _{i=1}^{n}(2J\phi _i + B)k_i \right\} \mathscr {D}\phi \Rightarrow \nonumber \\ Z= & {} \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \int _{-\infty }^{\infty } \left[ \sum \limits _{G \in \mathcal {G}} \exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \sum \limits _{i=1}^{n}(2J\phi _i + B)k_i \right\} \right] \mathscr {D}\phi \nonumber \\ \end{aligned}$$

(45)

In the above equation, the order of the integration and the sum of all the possible graphs in the model has been interchanged. It should also be noted that the term containing the sum of the squares of the fields $\phi _i$ is independent of the possible configurations of the graphs in the model and therefore Eq. 45 may take the following form

$$\begin{aligned} Z= & {} \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \int _{-\infty }^{\infty }\exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 \right\} \sum \limits _{G \in \mathcal {G}} \exp \left\{ \sum \limits _{i=1}^{n}(2J\phi _i + B)k_i \right\} \mathscr {D}\phi \nonumber \\ \end{aligned}$$

(46)

At this point, the sum of all possible model configurations has to be evaluated and the second term of the product within the integral of Eq. 46 is further analyzed as follows

$$\begin{aligned} \sum \limits _{i=1}^{n}(2J\phi _i + B)k_i= & {} \sum \limits _{i=1}^{n} (2J\phi _i + B)\sum \limits _{j=1}^{n} a_{ij} = \sum \limits _{i=1}^{n}\sum \limits _{j=1}^{n} (2J\phi _i + B)a_{ij} \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n} \left[ (2J\phi _i + B)a_{ij} + (2J\phi _j + B)a_{ji}\right] \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n} \left( 2J(\phi _i+\phi _j) + 2B\right) a_{ij} \end{aligned}$$

and

$$\begin{aligned}&\sum \limits _{G \in \mathcal {G}} \exp \left\{ \sum \limits _{i=1}^{n}(2J\phi _i + B)k_i \right\} = \prod _{i=1}^{n}\prod _{j=i+1}^{n} \sum \limits _{a_{ij} = 0}^{1} \exp \left\{ (2J(\phi _i+\phi _j) + 2B)a_{ij} \right\} \nonumber \\&\qquad = \prod _{i=1}^{n}\prod _{j=i+1}^{n} \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) = \exp \left\{ \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n}\ln \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) \right\} \nonumber \\ \end{aligned}$$

(47)

Substituting Eq. 47 into Eq. 46, yields

$$\begin{aligned} Z= & {} \left[ \frac{(n-1)J}{4\pi }\right] ^{\frac{n}{2}} \int _{-\infty }^{\infty }\exp \nonumber \\&\times \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n} \ln \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) \right\} \mathscr {D}\phi \Rightarrow \nonumber \\ Z= & {} \int _{-\infty }^{\infty }\exp \left\{ -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n} \ln \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) \right. \nonumber \\&\qquad \qquad \left. + \frac{n}{2}\ln [(n-1)J] - \frac{n}{2}\ln 4\pi \right\} \mathscr {D}\phi \nonumber \\ Z= & {} \int _{-\infty }^{\infty } \mathrm {e}^{\mathscr {H}(\phi )}\mathscr {D}\phi \end{aligned}$$

(48)

where $\mathscr {H}(\phi )$ is the effective hamiltonian

$$\begin{aligned} \mathscr {H}(\phi )= & {} -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \sum \limits _{i=1}^{n}\sum \limits _{j=i+1}^{n} \ln \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) \nonumber \\&+ \frac{n}{2}\ln (n-1)J - \frac{n}{2}\ln 4\pi \Rightarrow \nonumber \\ \mathscr {H}(\phi )= & {} -(n-1)J\sum \limits _{i=1}^{n}(\phi _i)^2 + \frac{1}{2}\sum \limits _{i=1}^{n}\sum \limits _{j=1(\ne i)}^{n} \ln \left( 1 + \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}\right) \nonumber \\&+ \frac{n}{2}\ln [(n-1)J] - \frac{n}{2}\ln 4\pi \end{aligned}$$

(49)

The importance of Eq. 48 above lies within the fact that it has been made possible to transform the initial model (Eq. 33) to a field theory of a continuous variable (evaluated at n points). Unfortunately, the integral of the equation above cannot be calculated in closed form (Park and Newman 2004b), but only through approximation techniques, like the mean-field theory.

1.1 Appendix 1.1: Mean-field theory

The simplest possible approximation is that of the mean-field, under which fluctuations in the field are ignored and $\phi _i$ is always set to its most probable value, located at the saddle point where the first derivative is equal to zero (Park and Newman 2004a).

$$\begin{aligned} \frac{\partial \mathscr {H}(\phi )}{\partial \phi _i} = 0 \Rightarrow -2(n-1)J\phi _i + J\sum \limits _{j=1(\ne j)}^{n} \frac{\mathrm {e}^{2J(\phi _i+\phi _j) + 2B}}{1+ \mathrm {e}^{2J(\phi _i+\phi _j) + 2B}} = 0 \end{aligned}$$

Using the identity

$$\begin{aligned} \frac{\mathrm {e}^{x}}{1 + \mathrm {e}^{x}} = \frac{1}{2}\left[ \tanh \frac{x}{2} + 1\right] \end{aligned}$$

(50)

the following equation emerges

$$\begin{aligned} - 2(n-1)\phi _i + \sum \limits _{j=1(\ne i)}^{n} \left[ \tanh (J(\phi _i+\phi _j) + B) +1 \right] = 0 \end{aligned}$$

which has the symmetric solution $\phi _0 = \phi _i$ for every i

$$\begin{aligned}&-2(n-1)\phi _0 + \sum \limits _{j=1(\ne i)}^{n} \left[ \tanh (2J\phi _0 + B) +1 \right] = 0 \Rightarrow \nonumber \\&-2(n-1)\phi _0 + (n-1)\left[ \tanh (2J\phi _0 + B) +1 \right] = 0 \Rightarrow \phi _0 \nonumber \\&\quad = \frac{1}{2}\left[ \tanh (2J\phi _0 + B) +1 \right] \end{aligned}$$

(51)

In this case, the path integral of Eq. 48 is simplified to n independent Gaussian Integrals, in which case the partition function becomes

$$\begin{aligned} Z = \int _{-\infty }^{\infty } \mathrm {e}^{\mathscr {H}(\phi )}\mathscr {D}\phi = \int _{-\infty }^{\infty } \mathrm {e}^{n\mathscr {H}(\phi _0)}\mathscr {D}\phi = \mathrm {e}^{n\mathscr {H}(\phi _0)} \int _{-\infty }^{\infty }\mathscr {D}\phi \Rightarrow Z = \mathrm {e}^{n\mathscr {H}(\phi _0)} \qquad \end{aligned}$$

(52)

and finally, the free energy, becomes

$$\begin{aligned} F \equiv \ln Z= & {} n\mathscr {H}(\phi _0) \Rightarrow \nonumber \\ F= & {} -n(n-1)J(\phi _0)^2 + \frac{1}{2}n(n-1) \ln \left( 1 + \mathrm {e}^{4J\phi _0 + 2B}\right) \nonumber \\&+ \frac{n}{2}\ln [(n-1)J] - \frac{n}{2}\ln 4\pi \end{aligned}$$

(53)

Appendix 2: Introducing ERGM into Bayesian NMF

Having chosen the likelihood function (Eq. 25) and the a-priori distribution (Eq. 24), we may employ the classic formula of Bayesian Inference (Eq. 6) to approximate the a-posteriori probability of the model parameters (elements of matrix $\widetilde{A}=WH$), given the data (elements of matrix A) and the hyper-parameters ($\Theta$)

$$\begin{aligned} P(\widetilde{a_{ij}}|a_{ij},\Theta ) \propto \mathcal {L}(a_{ij}|\widetilde{a_{ij}})&\times P(\widetilde{a_{ij}}|\Theta )\times P(\Theta ) \Rightarrow P(\widetilde{a_{ij}}|a_{ij},\Theta ) \propto \frac{\widetilde{a_{ij}}^{a_{ij}}}{a_{ij}!} \mathrm {e}^{-\widetilde{a_{ij}}} \nonumber \\&\times \frac{1}{Z} \times \mathrm {e}^{\Theta \widetilde{a_{i,j}}}\times P(\Theta ) \end{aligned}$$

(54)

The partition function Z is independent of the parameters $\widetilde{a_{ij}}$ and the data $a_{ij}$ of the model. Hyper-parameter $\Theta$ is also deterministically defined (Eq. 24). Therefore, Eq. 54 is simplified to

$$\begin{aligned} P(\widetilde{a_{ij}}|a_{ij},\Theta ) \propto \frac{\widetilde{a_{ij}}^{a_{ij}}}{a_{ij}!} \mathrm {e}^{-\widetilde{a_{ij}}} \times \mathrm {e}^{\Theta \widetilde{a_{i,j}}} \Rightarrow P(\widetilde{a_{ij}}|a_{ij},\Theta ) \propto \frac{\widetilde{a_{ij}}^{a_{ij}}}{a_{ij}!} \mathrm {e}^{(\Theta -1)\widetilde{a_{i,j}}} \end{aligned}$$

(55)

The element $\widetilde{a_{ij}}$ of matrix $\widetilde{A}$ is computed from the inner product of $i^{\text{ th }}$ row vector of matrix W times the $j^{\text{ th }}$ column vector of matrix H

$$\begin{aligned} P( \mathbf {w_i}^\top \mathbf {h_j}|a_{ij},\Theta ) \propto \frac{ \mathbf {w_i}^\top \mathbf {h_j}^{a_{ij}}}{a_{ij}!} \mathrm {e}^{(\Theta -1) \mathbf {w_i}^\top \mathbf {h_j}} \end{aligned}$$

(56)

The objective is to find those values for $\mathbf {w_i}^\top ,\mathbf {h_j}$ that maximize the a-posteriori probability of the parameters of the model (right-hand side of Eq. 56). As NMF has been defined as a minimization problem (Eq. 5), Eq. 56 above must be converted to an equivalent minimization problem. This conversion is achieved by taking the negative natural logarithm of the aforementioned equation (Wang and Zhang 2013)

$$\begin{aligned} \mathcal {D}(a_{ij}, \mathbf {w_i}^\top \mathbf {h_j}) \equiv -\ln P( \mathbf {w_i}^\top \mathbf {h_j}|a_{ij},\Theta ) = \ln (a_{ij}!) - a_{ij}\ln \mathbf {w_i}^\top \mathbf {h_j} + (1 - \Theta )\mathbf {w_i}^\top \mathbf {h_j} \end{aligned}$$

(57)

Using the Stirling Formula $\left( \ln (x!) = x\ln x - x\right)$ for the term $\ln (a_{ij}!)$, yields

$$\begin{aligned} \mathcal {D}(a_{ij}, \mathbf {w_i}^\top \mathbf {h_j})&= a_{ij}\ln a_{ij} - a_{ij}- a_{ij}\ln \mathbf {w_i}^\top \mathbf {h_j} + (1 - \Theta )\mathbf {w_i}^\top \mathbf {h_j} \nonumber \\&= a_{ij}\ln \frac{a_{ij}}{\mathbf {w_i}^\top \mathbf {h_j}} - a_{ij} + (1 - \Theta )\mathbf {w_i}^\top \mathbf {h_j} \end{aligned}$$

(58)

The gradient of Eq. 58 with respect to vectors $\mathbf {w_i}^\top , \mathbf {h_j}$ is computed as follows

$$\begin{aligned} \nabla \mathcal {D}_{\mathbf {w_i}^\top }(a_{ij}, \mathbf {w_i}^\top \mathbf {h_j})&= \sum \limits _{j=1}^{k}\left[ -\frac{a_{ij}}{\mathbf {w_i}^\top \mathbf {h_j}}\mathbf {h_j}^\top + (1 - \Theta )\mathbf {h_j}^\top \right] \nonumber \\&= -\frac{\mathbf {a_i}^\top }{\widetilde{\mathbf {a_i}}^\top }H^\top + (1 - \Theta )\mathbf {e}^\top H^\top \end{aligned}$$

(59)

$$\begin{aligned} \nabla \mathcal {D}_{\mathbf {h_j}}(a_{ij}, \mathbf {w_i}^\top \mathbf {h_j})&= \sum \limits _{i=1}^{k}\left[ -\mathbf {w_i}^\top \frac{a_{ij}}{\mathbf {w_i}^\top \mathbf {h_j}} + (1 - \Theta )\mathbf {w_i}^\top \right] \nonumber \\&= -W^\top \frac{\mathbf {a_i}^\top }{\widetilde{\mathbf {a_i}}^\top } + (1 - \Theta )W^\top \mathbf {e}^\top \end{aligned}$$

(60)

and vectors $\mathbf {w_i}^\top , \mathbf {h_j}$ are updated according to the multiplicative update rules below, where the update factor is the ratio of the negative component of the gradient versus the positive component (Lee and Seung 2000)

$$\begin{aligned} (\mathbf {w_i}^\top )^{(t+1)}&\leftarrow (\mathbf {w_i}^\top )^{(t)}\circ \frac{\nabla \mathcal {D}^{-}_{\mathbf {w^\top _{i}}}}{\nabla \mathcal {D}^{+}_{\mathbf {w^\top _{i}}}} \Rightarrow (\mathbf {w_i}^\top )^{(t+1)} \leftarrow (\mathbf {w_i}^\top )^{(t)}\circ \frac{\frac{\mathbf {a_i}^\top }{\widetilde{\mathbf {a_i}}^\top }H^\top }{(1 - \Theta )\mathbf {e}^\top H^\top } \nonumber \\ \mathbf {h_j}^{(t+1)}&\leftarrow \mathbf {h_j}^{(t)}\circ \frac{\nabla \mathcal {D}^{-}_{\mathbf {h_j}}}{\nabla \mathcal {D}^{+}_{\mathbf {h_{j}}}} \Rightarrow \mathbf {h_j}^{(t+1)} \leftarrow \mathbf {h_j}^{(t)}\circ \frac{W^\top \frac{\mathbf {a_i}^\top }{\widetilde{\mathbf {a_i}}^\top }}{(1 - \Theta )W^\top \mathbf {e}^\top } \end{aligned}$$

(61)

yielding to the following update rules for the basis and coefficient matrices W, H

$$\begin{aligned} W^{(t+1)} \leftarrow W^{(t)}\circ \frac{\frac{A}{WH}H^\top }{(1 - \Theta )EH^\top } \end{aligned}$$

(62)

$$\begin{aligned} H^{(t+1)} \leftarrow H^{(t)}\circ \frac{W^\top \frac{A}{WH}}{(1 - \Theta )W^\top E} \end{aligned}$$

(63)