Abstract
Most traditional collaborative filtering (CF) methods only use the user-item rating matrix to make recommendations, which usually suffer from cold-start and sparsity problems. To address these problems, on the one hand, some CF methods are proposed to incorporate auxiliary information such as user/item profiles; on the other hand, deep neural networks, which have powerful ability in learning effective representations, have achieved great success in recommender systems. However, these neural network based recommendation methods rarely consider the uncertainty of weights in the network and only obtain point estimates of the weights. Therefore, they maybe lack of calibrated probabilistic predictions and make overly confident decisions. To this end, we propose a new Bayesian dual neural network framework, named BDNet, to incorporate auxiliary information for recommendation. Specifically, we design two neural networks, one is to learn a common low dimensional space for users and items from the rating matrix, and another one is to project the attributes of users and items into another shared latent space. After that, the outputs of these two neural networks are combined to produce the final prediction. Furthermore, we introduce the uncertainty to all weights which are represented by probability distributions in our neural networks to make calibrated probabilistic predictions. Extensive experiments on real-world data sets are conducted to demonstrate the superiority of our model over various kinds of competitors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009, 452–461
Salakhutdinov R, Mnih A Probabilistic matrix factorization. In: Proceedings of the 20th In ternational conference on Advances in Neural Information Processing Systems. 2007, 1257–1264
Xue H J, Dai X, Zhang J, Huang S, Chen J. Deep matrix factorization models for recommender systems. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 3203–3209
Zhou Y, Wilkinson D, Schreiber R, Pan R. Large-scale parallel collaborative filtering for the netflix prize. In: Proceedings of the International Conference on Algorithmic Applications in Management. 2008, 337–348
Singh A P, Gordon G J. Relational learning via collective matrix factorization. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2008, 650–658
Ma H, Zhou D, Liu C, Lyu M R, King I. Recommender systems with social regularization. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining. 2011, 287–296
Singh A, Gordon G J. A Bayesian matrix factorization model for relational data. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence. 2010, 556–563
Gao Y, Wang X, Lei G, Chen Z. Learning to recommend with collaborative matrix factorization for new users. Journal of Computer Research and Development, 2017, 54(8): 1813–1823
Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18(7): 1527–1554
Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786): 504–507
Li S, Kawale J, Fu Y. Deep collaborative filtering via marginalized denoising auto-encoder. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015, 811–820
Wang H, Wang N, Yeung D Y. Collaborative deep learning for recommender systems. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015, 1235–1244
He X, Liao L, Zhang H, Nie L, Hu X, Chua T S. Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web. 2017, 173–182
Wang X, Wang Y. Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia. 2014, 627–636
Hernández-Lobato J M, Adams R. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In: Proceedings of the International Conference on Machine Learning. 2015, 1861–1869
Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D. Weight uncertainty in neural networks. In: Proceedings of the the 32nd International Conference on Machine Learning. 2015, 1613–1622
Bobadilla J, Ortega F, Hernando A, Gutiérrez A. Recommender systems survey. Knowledge-based Systems, 2013, 46(1): 109–132
Lang K. Newsweeder: learning to filter netnews. In: Proceedings of the International Conference on Machine Learning. 1995, 331–339
Chen T, Zhang W, Lu Q, Chen K, Zheng Z, Yu Y. SVDFeature: a toolkit for feature-based collaborative filtering. Journal of Machine Learning Research, 2012, 13(9): 3619–3622
Hu L, Cao J, Xu G, Cao L, Gu Z, Zhu C. Personalized recommendation via cross-domain triadic factorization. In: Proceedings of the 22nd International Conference on World Wide Web. 2013, 595–606
Agarwal D, Chen B C. Regression-based latent factor models. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2009, 19–28
Li W J, Yeung D Y, Zhang Z. Generalized latent factor models for social network analysis. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 2011, 1705–1710
Salakhutdinov R, Mnih A, Hinton G. Restricted boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine learning. 2007, 791–798
Georgiev K, Nakov P. A non-IID framework for collaborative filtering with restricted boltzmann machines. In: Proceedings of the International Conference on Machine Learning. 2013, 1148–1156
Graves A. Practical variational inference for neural networks. In: Proceedings of the 24th International conference on Neural Information Processing Systems. 2011, 2348–2356
Salakhutdinov R, Mnih A. Bayesian probabilistic matrix factorization using markov chain monte carlo. In: Proceedings of the the 25th International Conference on Machine Learning. 2008, 880–887
Acknowledgements
The research work was supported by the National Key R&D Program of China (2018YFB1004300), the National Natural Science Foundation of China (Grant Nos. 61773361, 61473273, 91546122), the Science and Technology Project of Guangdong Province (2015B010109005), the Project of Youth Innovation Promotion Association CAS (2017146). This work was also partly supported by the funding of WeChat cooperation project. We thank Bo Chen, Leyu Lin, Cheng Niu, Xiaohu Cheng for their constructive advices.
Author information
Authors and Affiliations
Corresponding author
Additional information
Jia He is a PhD candidate in Institute of Computing Technology, Chinese Academy of Sciences, China. Her research interests include machine learning, Bayesian nonparametric learning, multi-view learning. She has published several papers in some relevant research conferences, such as IJCAI, ECML, CIKM and ECAI.
Fuzhen Zhuang is an associate professor in the Institute of Computing Technology, Chinese Academy of Sciences, China. His research interests include transfer learning, machine learning, data mining, multi-task learning and recommendation systems. He has published more than 70 papers in some prestigious refereed journals and conference proceedings, such as IEEE Transactions on Knowledge and Data Engineering, IEEE Transactions on Cybernetics, ACM Transactions on Intelligent Systems and Technology, Information Sciences, IJCAI, AAAI, WWW, ICDE, ACM CIKM, ACM WSDM, SIAM SDM and IEEE ICDM.
Yanchi Liu is currently a PhD candidate in the Management Science and Information Systems Department at Rutgers, the State University of New Jersey. He received the PhD degree from the University of Science and Technology Beijing, and the BE degree from the Civil Aviation University of China. His research interests include data mining, business intelligence, urban computing, and recommender systems.
Qing He is a professor in the Institute of Computing Technology, Chinese Academy of Science (CAS), and he is a professor at the Graduate University of Chinese (GUCAS), China. He received the BS degree from Hebei Normal University, China in 1985, and the MS degree from Zhengzhou University, China in 1987, both in mathematics. He received the PhD degree in 2000 from Beijing Normal University in fuzzy mathematics and artificial intelligence, China. Since 1987 to 1997, he has been with Hebei University of Science and Technology. He is currently a doctoral tutor at the Institute of Computing and Technology, CAS. His interests include data mining, machine learning, classification, fuzzy clustering.
Fen Lin is currently a research scientist in Wechat, Tencent, and she received the PhD degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences, China. Her research interests include the research of algorithms and applications in nature language processing (NLP), data mining and artificial intelligence, especially the application of deep learning in NLP.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
He, J., Zhuang, F., Liu, Y. et al. Bayesian dual neural networks for recommendation. Front. Comput. Sci. 13, 1255–1265 (2019). https://doi.org/10.1007/s11704-018-8049-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-018-8049-1