Abstract
Link prediction aims to predict missing links or eliminate spurious links and anticipate new links by analyzing observed network topological structure information. Non-negative matrix factorization(NMF) is widely used in solving the issue of link prediction due to its good interpretability and scalability. However, most existing NMF-based approaches involve shallow decoder models, which are incapable of capturing complex hierarchical information hidden in networks, and seldom consider random noise. To address these issues, a novel deep autoencoder-like nonnegative matrix factorization method with \(\varvec{L_{2,1}}\) norm for link prediction models(DANMFL) is proposed. Unlike conventional NMF-based approaches, our model contains a decoder component and an encoder component, which capture complex hierarchical information effectively, leading to more accurate prediction results. In addition, we use the \(\varvec{L_{2,1}}\) norm to remove random noise in each layer and the convergence of our model is strictly proven. We conduct extensive experiments on twelve real-world networks and the experimental results show that DANMFL is superior to existing state-of-the-art baseline approaches in terms of prediction accuracy. Codes are available at https://github.com/litongf/DANMFL.










Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Yao Y, Cheng T, Li X et al (2023) Link prediction based on the mutual information with high-order clustering structure of nodes in complex networks. Physica A Stat Mech Appl 610:128428
Chai L, Tu L, Yu X et al (2023) Link prediction and its optimization based on low-rank representation of network structures. Expert Syst Appl 219:119680
Ai J, Cai Y, Su Z et al (2022) Predicting user-item links in recommender systems based on similarity-network resource allocation. Chaos, Solitons Fractals 158:112032
Su Z, Zheng X, Ai J et al (2020) Link prediction in recommender systems based on vector similarity. Physica A Stat Mech Appl 560:125154
Kumar A, Singh SS, Singh K et al (2020) Link prediction techniques, applications, and performance: a survey. Physica A Stat Mech Appl 553:124289
Cao YGJZSWGZ, Zhang (2019) A chaotic ant colony optimized link prediction algorithm. IEEE Trans Syst Man Cybern Syst PP(99):1–15
Lorrain F, White HC (1971) The structural equivalence of individuals in social networks. J Math Sociol 1(1):49–80
Adamic LA, Adar E (2003) Friends and neighbors on the web. Soc Networks 25(3):211–230
Zhou T, Lü L, Zhang YC (2009) Predicting missing links via local information. Eur Phys J B 71(4):623–630
Ahmad I, Akhtar MU, Noor S et al (2020) Missing link prediction using common neighbor and centrality based parameterized algorithm. Sci Rep 10(1):364
Zhou T, Lee YL, Wang G (2021) Experimental analyses on 2-hop-based and 3-hop-based link prediction algorithms. Physica A Stat Mech Appl 564:125532
Rafiee S, Salavati C, Abdollahpouri A (2020) CNDP: Link prediction based on common neighbors degree penalization. Physica A Stat Mech Appl 539:122950
Pech R, Hao D, Lee YL et al (2019) Link prediction via linear optimization. Physica A Stat Mech Appl 528:121319
Wang C, Satuluri V, Parthasarathy S (2007) Local probabilistic models for link prediction. Seventh ieee international conference on data mining (ICDM 2007) 322–331
Xzab C, Wpab C, Yx D (2018) An intermediary probability model for link prediction. Physica A Stat Mech Appl 512:902–912
Clauset CNMA, Moore (2008) Hierarchical structure and the prediction of missing links in networks. Nature 453(7191):98–101
Guimer R, Sales-Pardo M (2009) Missing and spurious interactions and the reconstruction of complex networks. Proc Natl Acad Sci 106(52):22073–22078
Kuang J, Scoglio C (2021) Layer reconstruction and missing link prediction of a multilayer network with maximum \(a posteriori\) estimation. Phys Rev E 104:024301
Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery, New York, NY, USA, KDD ’14, pp 701–710
Chen G, Xu C, Wang J et al (2019) Graph regularization weighted nonnegative matrix factorization for link prediction in weighted complex network. Neurocomputing 369(Dec.5):50–60
Wang W, Cai F, Jiao P et al (2016) A perturbation-based framework for link prediction via non-negative matrix factorization. Sci Rep 6(1):38938
Chen G, Xu C, Wang J et al (2020) Nonnegative matrix factorization for link prediction in directed complex networks using pagerank and asymmetric link clustering information. Expert Syst Appl 148:113290
Chen G, Xu C, Wang J et al (2020) Robust non-negative matrix factorization for link prediction in complex networks using manifold regularization and sparse learning. Physica A Stat Mech Appl 539:122882
Mahmoodi R, Seyedi SA, Akhlaghian Tab F et al (2023) Link prediction by adversarial nonnegative matrix factorization. Knowl-Based Syst 280:110998
Chen G, Wang H, Fang Y et al (2022) Link prediction by deep non-negative matrix factorization. Expert Syst Appl 188:115991
Zhou T (2021) Progresses and challenges in link prediction. IScience 24(11):103217
Jinyin Chen XW, Xu X (2022) Gc-lstm: graph convolution embedded lstm for dynamic network link prediction. Appl Intell 52:7513–7528
Le T, Le N, Le B (2023) Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Syst Appl 214:119122
Herlau T, Mørup M, Schmidt MN et al (2012) Detecting hierarchical structure in networks. In: 2012 3rd International workshop on cognitive information processing (CIP), 1–6
Newman MEJ (2018) Network structure from rich but noisy data. Nat Phys 14:542–545
Dong S, Wang P, Abbas K (2021) A survey on deep learning and its applications. Comput Sci Rev 40:100379
Ye F, Chen C, Zheng Z (2018) Deep autoencoder-like nonnegative matrix factorization for community detection. In: Proceedings of the 27th ACM international conference on information and knowledge management. Association for Computing Machinery, New York, NY, USA, CIKM ’18, pp 1393-1402
Zhang M (2020) Structural deep nonnegative matrix factorization for community detection. Appl Soft Comput 97:106846
De Handschutter P, Gillis N, Siebert X (2021) A survey on deep matrix factorizations. Comput Sci Rev 42:100423
Katz L (1953) A new status index derived from sociometric analysis. Psychometrika 18(1):39–43
Salton GMM (1983) Introduction to modern information retrieval. Inf Process Manage 19:402–403
Liu W, Lü L (2010) Link prediction based on local random walk. EPL (Europhysics Letters) 89(5):58007
Aziz F, Gul H, Muhammad I et al (2020) Link prediction using node information on local paths. Physica A Stat Mech Appl 557:124980
Leicht EA, Holme P, Newman MEJ (2006) Vertex similarity in networks. Phys Rev E Stat Nonlin Soft Matter Phys 73(2 Pt 2):026120
Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems 30(1):107–117. proceedings of the Seventh International World Wide Web Conference
Menon AK, Elkan C (2011) Link prediction via matrix factorization. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2011, Athens, Greece, September 5-9, 2011, Proceedings, Part II
Pech R, Hao D, Pan L, Cheng H, Zhou T (2017) Link prediction via matrix completion. EPL (Europhysics Letters) 117:38002
Wang W, Fei C, Jiao P et al (2016) A perturbation-based framework for link prediction via non-negative matrix factorization. Sci Rep 6:38938
Chen B, Li F, Chen S et al (2017) Link prediction based on non-negative matrix factorization. PLoS ONE 12(8):e0182968
Ma X, Sun P, Qin G (2017) Nonnegative matrix factorization algorithms for link prediction in temporal networks using graph communicability. Pattern Recognit 71:361–374
Wang W, Feng Y, Jiao P et al (2017) Kernel framework based on non-negative matrix factorization for networks reconstruction and link prediction. Knowl-Based Sys 137(dec1):104–114
Wang Z, Liang J, Li R (2018) A fusion probability matrix factorization framework for link prediction. Knowl-Based Sys 159:72–85
Akbar JA, Anthony K, Wu FX (2020) Mdipa: a microrna-drug interaction prediction approach based on non-negative matrix factorization. Bioinformatics 36:5061–5067
Du X, Yan J, Zhang R et al (2022) Cross-network skip-gram embedding for joint network alignment and link prediction. IEEE Trans Knowl Data Eng 34(3):1080–1095
Xiao Y, Li R, Lu X et al (2021) Link prediction based on feature representation and fusion. Inf Sci 548:1–17
Nasiri E, Berahmand K, Rostami M et al (2021) A novel link prediction algorithm for protein-protein interaction networks by attributed graph embedding. Comput Biol Med 137:104772
Barracchia EP, Pio G, Bifet A et al (2022) Lp-robin: Link prediction in dynamic networks exploiting incremental node embedding. Inf Sci 606:702–721
Zhou M, Han Q, Li M et al (2023) Nearest neighbor walk network embedding for link prediction in complex networks. Physica A Stat Mech Appl 620:128757
Lee D, Seung HS (2000) Algorithms for non-negative matrix factorization. In: Leen T, Dietterich T, Tresp V (eds) Advances in Neural Information Processing Systems, vol 13. MIT Press
Sun BJ, Shen H, Gao J, et al (2017) A non-negative symmetric encoder-decoder approach for community detection. Proceedings of the 2017 ACM on conference on information and knowledge management pp 597-606
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Arrar D, Kamel N, Lakhfif A (2023) A comprehensive survey of link prediction methods. The journal of supercomputing pp 1–41
Zhou T (2023) Discriminating abilities of threshold-free evaluation metrics in link prediction. Physica A Stat Mech Appl 615:128529
Rossi RA, Ahmed NK (2015) The network data repository with interactive graph analytics and visualization. AAAI https://networkrepository.com
Kunegis J (2013) Konect–the koblenz network collection. In Proc Int Conf on World Wide Web Companion 1343–1350
Zhou M, Jin H, Wu Q et al (2021) Betweenness centrality-based community adaptive network representation for link prediction. Appl Intell 52:3545–3558
Ding CH, Li T, Jordan MI (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55
Acknowledgements
This work is supported by Qinghai Province key research and development and transformation plan project(2022-GX-155); National Natural Science Fund:Research on Key Technologies of high-quality construction of Thangka and mural digital resources based on semi interaction(62262056);the National Natural Science Foundation of China (No. 62366030), the Gansu Provincial Natural Science Foundation (No.23JRRA8222), the Higher Education Innovation Fund project of Gansu (No. 2022A-022);Qinghai Key Laboratory of Internet of Things. Additionally, during the revision of this paper, we would like to express our sincere gratitude to Yangyang He and Puhe You for their invaluable experimental support and writing guidance.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A: Derivation for Eq. (17) and Eq. (22)
Appendix A: Derivation for Eq. (17) and Eq. (22)
In this part, we will analyse the convergence of DANMFL and give the theoretical proof. Next, we list the following theorems:
Theorem 1
Updating W using the rule of (17) while fixing the rest variables, the objective function L(W) monotonically decreases.
Theorem 2
Updating V using the rule of (22) while fixing the rest variables, the objective function L(V) monotonically decreases.
We utilize auxiliary function [54] to help us prove theorems 1 and 2.
Definition 3.1
\(G(h,h')\) is an auxiliary function for F(h) if the conditions in (A1) are satisfied.
The function is really helpful by reason of the following lemma.
Lemma 3.2
If G is an auxiliary function of L, then L is non-increasing under the update
Proof
\(L(h^{t+1})\le G(h^{t+1},h^{t})\le G(h^{t},h^{t})=L(h^{t})\) \(\square \)
Lemma 3.3
Let \(A\in \mathbb {R}^{n\times n}\) is an any nonnegative matrix, \(B\in \mathbb {R}^{k\times k}\), \(S \in \mathbb {R}^{n\times k}\), \(S'\in \mathbb {R}^{n\times k}\), A and B are symmetric, then the following inequality holds [62]:
Here, we firstly prove the theorem 1 and rewrite the objective function L with removing unrelated elements.
According to the formulas: \(Tr(A^T)=Tr(A)\) and \(Tr(AB) \) \( =Tr(BA)\), the (A4) can be further described as following:
Let
be an auxiliary function of \(L(W_{i})\). Obviously, when \((W_{i})=(W_{i})'\), \(G((W_{i}),(W_{i})')=L(W_{i})\). Therefore, we need to prove \(G((W_{i}),(W_{i})')\le L(W_{i})\) here. When \((W_{i})')\ne L(W_{i})\), according to Lemma 3.3 and inequation \(z\ge 1+\log z\), we get the following inequations:
Summing all above bounds and we get the following inequations:
Hence, \(G((W_{i}),(W_{i})')\) is an auxiliary function of \(L(W_{i})\). To get the global minimum value of function \(G((W_{i}),(W_{i})')\), we fix the \(W_{i}'\) and take the derivative of \(G((W_{i}),(W_{i})')\) on \((W_{i})_{kl}\).
Because the Hessian matrix \(\frac{\partial ^{2} G((W_{i}),(W_{i})_{kl}')}{\partial (W_{i})_{kl}\partial ( W_{i})_{kl}}\) is positive define, the \(G((W_{i}),(W_{i})')\) is a convex function. Setting \(\frac{\partial G((W_{i}),(W_{i})_{kl}')}{\partial ( W_{i})_{kl})}\)=0 and based on the KTT conditions, we get the following updating rules:
Noting \(W^{t+1}\leftarrow W\) and \(W^{t}\leftarrow W'\), the (A10) recovers the (17).Thus, the objective function L monotonically decreases according to the updating rule. Equation (17) for L.
Next, we will prove the theorem 2. Let us rewrite the (18) here:
According to the relationship between matrix norm and matrix trace, we rewrite the equation L with removing irrelevant elements:
Now an auxiliary function of (A12) is
When \((V_{i})'=(V_{i})\), clearly \(G(V_{i},V_{i}')=L(V_{i})\). Hence, we need to prove \(G(V_{i},V_{i}')>L(V_{i})\) here. Based on Lemma 3.3 and \(z\ge 1+\log z\), we get the following inequations:
The first term of (A13) is always bigger than the fourth term of (A12) and the fourth term of (A13) is always smaller than the first term of (A12). Therefore, we get \(G((V_{i}),(V_{i})')>L(V_{i})\). Our objective now is to obtain the global minimum value of \(G((V_{i}),(V_{i}'))\). The derivative of \(G((V_{i}),(V_{i})')\) is
Because the Hessian matrix \(\frac{\partial ^{2} G((V_{i}),(V_{i})_{kl}')}{\partial (V_{i})_{kl}\partial ( V_{i})_{kl}}\) is positive define and the \(G((V_{i}),(V_{i})')\) is a convex function. Setting \(\frac{\partial G((V_{i}),(V_{i})_{kl}')}{\partial ( V_{i})_{kl})}\)=0 and according to the KTT condition, we get the following updating rules:
Noting \(V^{t+1}\leftarrow V\) and \(V^{t}\leftarrow V'\), the (A16) recovers the (22). Therefore, objective function L monotonically decreases under the update rule (22) for L.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, T., Zhang, R., Yao, Y. et al. Link prediction using deep autoencoder-like non-negative matrix factorization with L21-norm. Appl Intell 54, 4095–4120 (2024). https://doi.org/10.1007/s10489-024-05365-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05365-6