ABSTRACT
As a fundamental task, link prediction has pervasive applications in social networks, webpage networks, information retrieval and bioinformatics. Among link prediction methods, latent variable models, such as relational topic model and its variants, which jointly model both network structure and node attributes, have shown promising performance for predicting network structures and discovering latent representations. However, these methods are still limited in their representation learning capability from high-dimensional data or consider only text modality of the content. Thus they are very limited in current multimedia scenario. This paper proposes a Bayesian deep generative model called relational variational autoencoder (RVAE) that considers both links and content for link prediction in the multimedia scenario. The model learns deep latent representations from content data in an unsupervised manner, and also learns network structures from both content and link information. Unlike previous deep learning methods with denoising criteria, the proposed RVAE learns a latent distribution for content in latent space, instead of observation space, through an inference network, and can be easily extended to multimedia modalities other than text. Experiments show that RVAE is able to significantly outperform the state-of-the-art link prediction methods with more robust performance.
- Edoardo M. Airoldi, David M. Blei, Stephen E. Fienberg, and Eric P. Xing: 2008: Mixed membership stochastic blockmodels: Journal of Machine Learning Research Vol. 9, Sep (2008), 1981--2014. Google ScholarDigital Library
- Mohammad Al Hasan, Vineet Chaoji, Saeed Salem, and Mohammed Zaki: 2006: Link prediction using supervised learning. In SDM06: Workshop on Link Analysis, Counter-terrorism and Security.Google Scholar
- David M. Blei, Andrew Y. Ng, and Michael I. Jordan: 2003: Latent dirichlet allocation: Journal of Machine Learning Research Vol. 3, Jan (2003), 993--1022. Google ScholarDigital Library
- Jonathan Chang and David M. Blei: 2010: Hierarchical relational models for document networks: The Annals of Applied Statistics (2010), 124--150.Google Scholar
- Ning Chen, Jun Zhu, Fei Xia, and Bo Zhang: 2015: Discriminative relational topic models: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, 5 (2015), 973--986.Google ScholarCross Ref
- Janardhan Rao Doppa, Jun Yu, Prasad Tadepalli, and Lise Getoor: 2009: Chance-constrained programs for link prediction: NIPS Workshop on Analyzing Networks and Learning with Graphs.Google Scholar
- Anna Goldenberg, Alice X. Zheng, Stephen E. Fienberg, Edoardo M. Airoldi, and others: 2010: A survey of statistical network models: Foundations and Trends® in Machine Learning, Vol. 2, 2 (2010), 129--233. Google ScholarDigital Library
- Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber: 2006: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning. ACM, 369--376. Google ScholarDigital Library
- Peter D. Hoff, Adrian E. Raftery, and Mark S. Handcock: 2002: Latent space approaches to social network analysis: J. Amer. Statist. Assoc. Vol. 97, 460 (2002), 1090--1098.Google ScholarCross Ref
- Jake M. Hofman and Chris H. Wiggins: 2008: Bayesian approach to network modularity: Physical Review Letters Vol. 100, 25 (2008), 258701.Google ScholarCross Ref
- David Hunter, Padhraic Smyth, Duy Q. Vu, and Arthur U. Asuncion: 2011: Dynamic egocentric models for citation networks: Proceedings of the 28th International Conference on Machine Learning (ICML-11). 857--864. Google ScholarDigital Library
- Charles Kemp, Thomas L. Griffiths, and Joshua B. Tenenbaum: 2004: Discovering latent classes in relational data: (2004).Google Scholar
- Diederik P. Kingma and Max Welling: 2013: Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations.Google Scholar
- Daphne Koller and Nir Friedman: 2009: Probabilistic Graphical Models: Principles and Techniques. MIT press. Google ScholarDigital Library
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton: 2015: Deep learning: Nature, Vol. 521, 7553 (2015), 436--444.Google Scholar
- Jure Leskovec, Jon Kleinberg, and Christos Faloutsos: 2005: Graphs over time: densification laws, shrinking diameters and possible explanations.In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining. ACM, 177--187. Google ScholarDigital Library
- Xiaoyi Li, Nan Du, Hui Li, Kang Li, Jing Gao, and Aidong Zhang: 2014: A deep learning approach to link prediction in dynamic networks Proceedings of the 2014 SIAM International Conference on Data Mining. SIAM, 289--297.Google Scholar
- Ryan N. Lichtenwalter, Jake T. Lussier, and Nitesh V. Chawla: 2010: New perspectives and methods in link prediction: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 243--252. Google ScholarDigital Library
- Kurt Miller, Michael I. Jordan, and Thomas L. Griffiths: 2009: Nonparametric latent feature models for link prediction Advances in Neural Information Processing Systems. 1276--1284. Google ScholarDigital Library
- Kevin P. Murphy: 2012: Machine Learning: A Probabilistic Perspective. MIT press. Google ScholarDigital Library
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena: 2014: Deepwalk: Online learning of social representations Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: ACM, 701--710. Google ScholarDigital Library
- Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin: 2016: Variational autoencoder for deep learning of images, labels and captions Advances in Neural Information Processing Systems. 2352--2360.Google Scholar
- Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, and Lawrence Carin: 2016: A deep generative deconvolutional image model. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. 741--750.Google Scholar
- Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol: 2010: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion: Journal of Machine Learning Research Vol. 11, Dec (2010), 3371--3408. Google ScholarDigital Library
- Chong Wang and David M. Blei: 2011: Collaborative topic modeling for recommending scientific articles Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: ACM, 448--456. Google ScholarDigital Library
- Daixin Wang, Peng Cui, and Wenwu Zhu: 2016: Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1225--1234. Google ScholarDigital Library
- Hao Wang, Binyi Chen, and Wu-Jun Li: 2013: Collaborative topic regression with social regularization for tag recommendation: International Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- Hao Wang, Xingjian Shi, and Dit-Yan Yeung: 2017: Relational Deep Learning: A Deep Latent Variable Model for Link Prediction Proceedings of the AAAI.Google Scholar
- Zhaoquan Yuan, Jitao Sang, Yan Liu, and Changsheng Xu: 2013: Latent feature learning in social media network: Proceedings of the 21st ACM International Conference on Multimedia. ACM, 253--262. Google ScholarDigital Library
Index Terms
- Relational Variational Autoencoder for Link Prediction with Multimedia Data
Recommendations
Collaborative Variational Autoencoder for Recommender Systems
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningModern recommender systems usually employ collaborative filtering with rating information to recommend items to users due to its successful performance. However, because of the drawbacks of collaborative-based methods such as sparsity, cold start, etc., ...
Deep active inference
This work combines the free energy principle and the ensuing active inference dynamics with recent advances in variational inference in deep generative models, and evolution strategies to introduce the "deep active inference" agent. This agent minimises ...
Variational Diffusion Autoencoders with Random Walk Sampling
Computer Vision – ECCV 2020AbstractVariational autoencoders (VAEs) and generative adversarial networks (GANs) enjoy an intuitive connection to manifold learning: in training the decoder/generator is optimized to approximate a homeomorphism between the data distribution and the ...
Comments