Robust asymmetric non-negative matrix factorization for clustering nodes in directed networks

Abstract

Directed networks appear in an expanding array of applications, for example, the world wide web, social networks, transaction networks, and citation networks. A critical task in analyzing directed networks is clustering, where the goal is partitioning the network's nodes based on their similarities while accounting for the direction of relationships between nodes. Non-negative matrix factorization (NMF) and its variations have been used to cluster the nodes in directed networks by approximating their adjacency matrices efficaciously. The differences between the corresponding entries of the actual and approximate adjacency matrices are considered as errors, which are assumed to follow Gaussian distributions. However, these errors could deviate from Gaussian distributions in various real-world networks. In this work, we propose a robust asymmetric non-negative matrix factorization method to cluster the nodes in directed networks. Recognizing that the errors do not follow Gaussian distributions in real-world networks, the proposed method assumes that the errors follow a Cauchy distribution, which resembles the Gaussian distribution but has heavier tails. Experiments using real-world as well as artificial networks show that the proposed method outperforms existing NMF methods and other representative work in clustering in various settings.

Appendix

To show that updating rules (5) and (6) are correct, the Lagrangian objective function is:

$$ \begin{aligned} L & = \mathop \sum \limits_{i = 1}^{n} ln\left( {\left\| {{\varvec{a}}_{i} - \left( {{\mathbf{WHW}}^{{\text{T}}} } \right)_{i} } \right\|^{2} + \gamma^{2} } \right) - tr\left( {{{\varvec{\upbeta}}}_{1} {\mathbf{W}}^{T} } \right) - tr\left( {{{\varvec{\upbeta}}}_{2} {\mathbf{H}}^{T} } \right),\;{{\varvec{\upbeta}}}_{1} \in {\mathbb{R}}_{ + }^{n \times r} ,\;{{\varvec{\upbeta}}}_{2} \in {\mathbb{R}}_{ + }^{r \times r} \\ \frac{\partial L}{{\partial {\mathbf{W}}}} & = \mathop \sum \limits_{i = 1}^{n} \frac{1}{{\left\| {{\varvec{a}}_{i} - \left( {{\mathbf{WHW}}^{{\text{T}}} } \right)_{i} } \right\|^{2} + \gamma^{2} }}\frac{{\partial \left\| {{\varvec{a}}_{i} - \left( {{\mathbf{WHW}}^{{\text{T}}} } \right)_{i} } \right\|^{2} + \gamma^{2} }}{\partial W} - {{\varvec{\upbeta}}}_{1} \\ & = \frac{{\partial \left\| {{\mathbf{AD}} - {\mathbf{WHW}}^{{\text{T}}} {\mathbf{D}} } \right\|^{2} + \gamma^{2} }}{{\partial {\mathbf{W}}}} - {{\varvec{\upbeta}}}_{1} \;{\text{where}}\; \left[ {\mathbf{D}} \right]_{ii} = \frac{1}{{\left\| {{\varvec{a}}_{i} - \left( {{\mathbf{WHW}}^{{\text{T}}} } \right)_{i} } \right\|^{2} + \gamma^{2} }}. \\ \end{aligned} $$

Note that $\frac{1}{{\parallel {{\varvec{a}}}_{i}- {\left(\mathbf{W}\mathbf{H}{\mathbf{W}}^{{\text{T}}}\right)}_{i}\parallel }^{2}+{\gamma }^{2}}$ is outside the derivative, so it will be treated as constant.

$$ \begin{aligned} \frac{\partial L}{{\partial {\mathbf{W}}}} & = \frac{{\partial tr\left[ {\left( {{\mathbf{AD}} - {\mathbf{WHW}}^{{\text{T}}} {\mathbf{D}}} \right)\left( {{\mathbf{AD}} - {\mathbf{WHW}}^{{\text{T}}} {\mathbf{D}}} \right)^{T} } \right]}}{{\partial {\mathbf{W}}}} - {{\varvec{\upbeta}}}_{1} \\ & = 2\left( {{\mathbf{WHW}}^{{\text{T}}} {\mathbf{DWH}}^{{\text{T}}} + {\mathbf{DWH}}^{{\text{T}}} {\mathbf{W}}^{{\text{T}}} {\mathbf{WH}} - {\mathbf{ADWH}}^{{\text{T}}} - {\mathbf{DA}}^{{\text{T}}} {\mathbf{WH}}} \right) - {{\varvec{\upbeta}}}_{1} \\ \frac{\partial L}{{\partial {\mathbf{H}}}} & = 2\left( {{\mathbf{W}}^{{\text{T}}} {\mathbf{WHW}}^{{\text{T}}} {\mathbf{DW}} - {\mathbf{W}}^{{\text{T}}} {\mathbf{ADW}}} \right) - {{\varvec{\upbeta}}}_{2} \\ \end{aligned} $$

$$ \begin{aligned} \frac{\partial L}{{\partial {\mathbf{W}}}} & = 0 \Rightarrow \left[ {{{\varvec{\upbeta}}}_{1} } \right]_{ik} = 2\left[ {{\mathbf{WHW}}^{{\text{T}}} {\mathbf{DWH}}^{{\text{T}}} + {\mathbf{DWH}}^{{\text{T}}} {\mathbf{W}}^{{\text{T}}} {\mathbf{WH}} - {\mathbf{ADWH}}^{{\text{T}}} - {\mathbf{DA}}^{{\text{T}}} {\mathbf{WH}}} \right]_{ik} \\ \frac{\partial L}{{\partial {\mathbf{H}}}} & = 0 \Rightarrow [{{\varvec{\upbeta}}}_{2} ]_{kj} = 2\left[ {{\mathbf{W}}^{{\text{T}}} {\mathbf{WHW}}^{{\text{T}}} {\mathbf{DW}} - {\mathbf{W}}^{{\text{T}}} {\mathbf{ADW}}} \right]_{kj} \\ \end{aligned} $$

Complementary slackness means that ${{[{\varvec{\upbeta}}}_{1}]}_{ik}[{\mathbf{W}]}_{ik}=0$ and ${{[{\varvec{\upbeta}}}_{2}]}_{kj}{[\mathbf{H}]}_{kj}=0$.

Thus, if $\mathbf{W}$ and $\mathbf{H}$ satisfy the KKT conditions, updated $\mathbf{W}$ and $\mathbf{H}$ using Eqs. (5) and (6) also satisfy the KKT conditions.