Skip to main content
SpringerLink
Log in

Variational autoencoder Bayesian matrix factorization (VABMF) for collaborative filtering

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Probabilistic matrix factorization (PMF) is the most popular method among low-rank matrix approximation approaches that address the sparsity problem in collaborative filtering for recommender systems. PMF depends on the classical maximum a posteriori estimator for estimating model parameters; however, these approaches are vulnerable to overfitting because of the nature of a single point estimation that is pursued by these models. An alternative approach to PMF is a Bayesian PMF model that suggests the Markov chain Monte Carlo algorithm as a full estimation for approximate intractable posterior over model parameters. However, despite its success in increasing prediction, it has a high computational cost. To this end, we proposed a novel Bayesian deep learning-based model treatment, namely, variational autoencoder Bayesian matrix factorization (VABMF). The proposed model uses stochastic gradient variational Bayes to estimate intractable posteriors and expectation–maximization-style estimators to learn model parameters. The model was evaluated on the basis of three MovieLens datasets, namely, Ml-100k, Ml-1M, and Ml-10M. Experimental results showed that our proposed VABMF model significantly outperforms state-of-the-art RS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ahmadian S, Afsharchi M, Meghdadi M (2019) A novel approach based on multi-view reliability measures to alleviate data sparsity in recommender systems. Multimedia Tools and Applications 78(13):17763–17798

    Article  Google Scholar 

  2. Su X, Khoshgoftaar T M (2009) A survey of collaborative filtering techniques. adv Artif Intell 2009

  3. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749

    Article  Google Scholar 

  4. Ricci F, Rokach L, Shapira B (2011) Introduction to recommender systems handbook. In: Recommender systems handbook, Springer, pp 1–35

  5. Ahmadian S, Afsharchi M, Meghdadi M (2019) An effective social recommendation method based on user reputation model and rating profile enhancement. J Inf Sci 45(5):607–642

    Article  Google Scholar 

  6. Ahmadian S, Meghdadi M, Afsharchi M (2018) Incorporating reliable virtual ratings into social recommendation systems. Appl Intell 48(11):4448–4469

    Article  Google Scholar 

  7. Sachan A, Richariya V (2013) A survey on recommender systems based on collaborative filtering technique. International Journal of Innovations in Engineering and Technology (IJIET) 2(2):8–14

    Google Scholar 

  8. Yu C, Huang L (2017) Clucf: a clustering cf algorithm to address data sparsity problem. SOCA 11(1):33–45

    Article  Google Scholar 

  9. Singh M (2020) Scalability and sparsity issues in recommender datasets: a survey. Knowl Inf Syst 1–43

  10. Jalili M, Ahmadian S, Izadi M, Moradi P, Salehi M (2018) Evaluating collaborative filtering recommender algorithms: a survey. IEEE Access 6:74003–74024

    Article  Google Scholar 

  11. Salakhutdinov R, Mnih A (2008) Probabilistic matrix factorization. In: Advances in neural information processing systems 20 - proceedings of the 2007 conference, pp 1–8

  12. Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using markov chain monte carlo. In: Proceedings of the 25th international conference on machine learning. https://doi.org/10.1145/1390156.1390267, pp 880–887

  13. Ahmadian S, Meghdadi M, Afsharchi M (2018) A social recommendation method based on an adaptive neighbor selection mechanism. Inf Process Manag 54(4):707–725

    Article  Google Scholar 

  14. Ahmadian S, Joorabloo N, Jalili M, Ren Y, Meghdadi M, Afsharchi M (2020) A social recommender system based on reliable implicit relationships. Knowl-Based Syst 192:105371

    Article  Google Scholar 

  15. Ahmadian S, Joorabloo N, Jalili M, Meghdadi M, Afsharchi M, Ren Y (2018) A temporal clustering approach for social recommender systems. In: 2018 IEEE/ACM International conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 1139–1144

  16. Moradi P, Rezaimehr F, Ahmadian S, Jalili M (2016) A trust-aware recommender algorithm based on users overlapping community structure. In: 2016 Sixteenth international conference on advances in ICT for emerging regions (ICTer), IEEE, pp 162–167

  17. Ahmadian S, Moradi P, Akhlaghian F (2014) An improved model of trust-aware recommender systems using reliability measurements. In: 2014 6th conference on information and knowledge technology (IKT), IEEE, pp 98–103

  18. Lu J, Wu D, Mao M, Wang W, Zhang G (2015) Recommender system application developments: a survey. Decis Support Syst 74:12–32

    Article  Google Scholar 

  19. Rahmani HA, Aliannejadi M, Ahmadian S, Baratchi M, Afsharchi M, Crestani F (2019) Lglmf: local geographical based logistic matrix factorization model for poi recommendation. In: Asia information retrieval symposium, Springer, pp 66–78

  20. Gong S, Ye H, Tan H (2009) Combining memory-based and model-based collaborative filtering in recommender system. In: 2009 Pacific-Asia conference on circuits, communications and systems, IEEE, pp 690–693

  21. Marlin B, Zemel R S (2004) The multiple multiplicative factor model for collaborative filtering. In: Proceedings, twenty-first international conference on machine learning, ICML 2004. https://doi.org/10.1145/1015330.1015437, p 73

  22. Beal M J (2003) Variational algorithms for approximate bayesian inference. Phd Thesis, UCL (University College London)

  23. Kingma D P, Welling M (2013) Auto-encoding variational bayes, pp 1–14. arXiv:1312.6114

  24. Hoffman M D, Blei D M, Wang C, Paisley J (2013) Stochastic variational inference. J Mach Learn Res 14(1):1303–1347

    MathSciNet  MATH  Google Scholar 

  25. Brooks S, Gelman A, Jones G, Meng X L (2011) Handbook of Markov chain Monte Carlo. CRC Press, Boca Raton

    Book  Google Scholar 

  26. Minka T P (2013) Expectation propagation for approximate bayesian inference, pp 362–369. arXiv:1301.2294

  27. Boykov Y, Veksler O, Zabih R (1998) Markov random fields with efficient approximations. In: Proceedings. 1998 IEEE computer society conference on computer vision and pattern recognition (Cat. No. 98CB36231), IEEE, pp 648–655

  28. Nielsen TD, Jensen FV (2009) Bayesian networks and decision graphs. Springer Science & Business Media

  29. Blei D M, Kucukelbir A, McAuliffe J D (2017) Variational inference: a review for statisticians. J Am Stat Assoc 112(518):859–877

    Article  MathSciNet  Google Scholar 

  30. Korattikara A, Chen Y, Welling M (2014) Austerity in mcmc land: cutting the metropolis-hastings budget. In: International conference on machine learning, pp 181–189

  31. Dziugaite GK, Roy DM (2015) Neural network matrix factorization. Conference paper at ICLR 2016, pp 1–7. arXiv:1511.06443

  32. Kampman O, Heaukulani C (2019) Variational inference for neural network matrix factorization and its application to stochastic blockmodeling. ICML 2019 workshop: learning and reasoning with graph-structured representations. arXiv:https://arxiv.org/abs/1905.04502v3

  33. Xiao T, Shen H (2019) Neural variational matrix factorization for collaborative filtering in recommendation systems. Appl Intell 49(10):3558–3569. https://doi.org/10.1007/s10489-019-01469-6

    Article  Google Scholar 

  34. Yıldırım E, Azad P, Öğüdücü ŞG (2020) Neural hybrid recommender: recommendation needs collaboration. In: International workshop on new frontiers in mining complex patterns, Springer, pp 52–66. https://doi.org/10.1007/978-3-030-48861-1_4, arXiv:https://arxiv.org/abs/1909.13330

  35. Karamanolakis G, Yuan J, Cherian KR, Tang D, Narayan AR, Jebara T (2018) Item recommendation with variational autoencoders and heterogeneous priors. In: ACM International conference proceeding series. https://doi.org/10.1145/3270323.3270329, pp 10–14

  36. Wu Y, DuBois C, Zheng AX, Ester M (2016) Collaborative denoising autoencoders for top-n recommender systems. In: WSDM 2016 - Proceedings of the 9th ACM international conference on web search and data mining, pp 153–162

  37. Dong B, Zhu Y, Li L, Wu X (2020) Hybrid collaborative recommendation via dual-autoencoder. IEEE Access 8:46030–46040

    Article  Google Scholar 

  38. Wang H, Wang N, Yeung DY (2015) Collaborative deep learning for recommender systems. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining 2015-August. https://doi.org/10.1145/2783258.2783273, pp 1235–1244

  39. Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: International conference on information and knowledge management, proceedings 19-23-Oct. https://doi.org/10.1145/2806416.2806527, pp 811–820

  40. Liang D, Krishnan RG, Hoffman MD, Jebara T (2019) Variational autoencoders for collaborative filtering. In: Proceedings of the 2018 world wide web conference, pp 600–605. https://doi.org/10.1145/3289600.3291007

  41. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  42. Xue HJ, Dai XY, Zhang J, Huang S, Chen J (2017) Deep matrix factorization models for recommender systems. In: IJCAI International joint conference on artificial intelligence. https://doi.org/10.24963/ijcai.2017/447, pp 3203–3209

  43. Liu T, Tao D (2015) On the performance of manhattan nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 27(9):1851–1863

    Article  MathSciNet  Google Scholar 

  44. Wang S, Tang J, Wang Y, Liu H (2018) Exploring hierarchical structures for recommender systems. IEEE Trans Knowl Data Eng 30(6):1022–1035

    Article  Google Scholar 

  45. Mongia A, Jhamb N, Chouzenoux E, Majumdar A (2020) Deep latent factor model for collaborative filtering. Signal Process 169:107366

    Article  Google Scholar 

  46. Deerwester S, Dumais S T, Furnas G W, Landauer T K, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391–407

    Article  Google Scholar 

  47. Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 426–434

  48. Feng C, Liang J, Song P, Wang Z (2020) A fusion collaborative filtering method for sparse data in recommender systems. Inf Sci 521:365–379

    Article  MathSciNet  Google Scholar 

  49. Bowman SR, Vilnis L, Vinyals O, Dai AM, Jozefowicz R, Bengio S (2015) Generating sentences from a continuous space. In: CoNLL 2016 - 20th SIGNLL conference on computational natural language learning, proceedings, pp 10–21. https://doi.org/10.18653/v1/k16-1002

Download references

Acknowledgments

The authors would like to thank the anonymous referees for their helpful comments and suggestions. This study was partially supported by the National Natural Science Foundation of China (Z201G10110G20003).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Central South University, Changsha, 410083, China

    Ali Aldhubri, Yu Lasheng & Farida Mohsen

  2. School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China

    Majjed Al-Qatf

Authors
  1. Ali Aldhubri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Lasheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farida Mohsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Majjed Al-Qatf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Lasheng.

Ethics declarations

Conflict of interests

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). He is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from (yulasheng@csu.edu.cn) Signed by all authors as follows:

Ali M. Aldhubri,

Yu lashing,

Farida Mohsen, and

Majjed Al-Qatf

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aldhubri, A., Lasheng, Y., Mohsen, F. et al. Variational autoencoder Bayesian matrix factorization (VABMF) for collaborative filtering. Appl Intell 51, 5132–5145 (2021). https://doi.org/10.1007/s10489-020-02049-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02049-9

Keywords

Search

Navigation