Abstract
Recommender systems play an important role with many applications in natural language processing such as in e-commerce services. Matrix factorization (MF) is a powerful method in recommender systems, but a main issue is the sparse data problem. In order to overcome the problem, some previous models use neural networks to represent additional information such as product item reviews to enhance MF-based methods, and obtain improvement in recommender systems. However, these models use conventional pre-trained word embeddings, which raise a question whether recent powerful models such as BERT can improve these MF-based methods enhanced by item reviews. In this work, we investigate the effect of utilizing BERT model to improve some previous models, especially focusing on several specific sparse data settings. Experimental results on the MovieLens dataset show that our model has successfully utilized BERT to represent item reviews and outperformed the previous probabilistic MF-based model which does not use item reviews. We also conducted intensive analyses on several settings related to sparse data and obtained some promising findings related to the lengths of review texts, which may open directions to improve this on-going model to solve the problem of sparse data in MF-based recommender systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We set the maximum length of the concatenated sequences as 300, which we followed [7].
- 2.
- 3.
- 4.
- 5.
References
Dahdouh, K., Dakkak, A., Oughdir, L., Ibriz, A.: Large-scale e-learning recommender system based on spark and Hadoop. J. Big Data 6(1), 1–23 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint: arXiv:1810.04805 (2018)
Felfernig, A., Le, V.M., Popescu, A., Uta, M., Tran, T.N.T., Atas, M.: An overview of recommender systems and machine learning in feature modeling and configuration. In: 15th International Working Conference on Variability Modelling of Software-Intensive Systems, pp. 1–8 (2021)
Feng, C., Liang, J., Song, P., Wang, Z.: A fusion collaborative filtering method for sparse data in recommender systems. Inf. Sci. 521, 365–379 (2020)
Grießhaber, D., Maucher, J., Vu, N.T.: Fine-tuning BERT for low-resource natural language understanding via active learning. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 1158–1171 (2020)
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web, pp. 173–182 (2017)
Kim, D.H., Park, C., Oh, J., Lee, S., Yu, H.: Convolutional matrix factorization for document context-aware recommendation. In: Sen, S., Geyer, W., Freyne, J., Castells, P. (eds.) RecSys, pp. 233–240. ACM (2016)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
Lika, B., Kolomvatsos, K., Hadjiefthymiades, S.: Facing the cold start problem in recommender systems. Expert Syst. Appl. 41(4), 2065–2073 (2014)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint: arXiv:1907.11692 (2019)
Nguyen, H.X., Nguyen, M.L.: Attention mechanism for recommender systems. In: Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation. Association for Computational Linguistics, Japan (2019)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of EMNLP, pp. 1532–1543 (2014)
Rendle, S., Krichene, W., Zhang, L., Anderson, J.: Neural collaborative filtering vs. matrix factorization revisited. In: Fourteenth ACM Conference on Recommender Systems, pp. 240–248 (2020)
Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Advances in Neural Information Processing Systems, vol. 20 (2008)
Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: Apté, C., Ghosh, J., Smyth, P. (eds.) KDD, pp. 448–456. ACM (2011)
Wang, H., Wang, N., Yeung, D.Y.: Collaborative deep learning for recommender systems. In: Cao, L., Zhang, C., Joachims, T., Webb, G.I., Margineantu, D.D., Williams, G. (eds.) KDD, pp. 1235–1244. ACM (2015)
Wang, Z., Karthikeyan, K., Mayhew, S., Roth, D.: Extending multilingual BERT to low-resource languages. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 2649–2656 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, X.H., Trieu, L.H., Nguyen, L.M. (2023). Investigating the Effects of Pre-trained BERT to Improve Sparse Data Recommender Systems. In: Yada, K., Takama, Y., Mineshima, K., Satoh, K. (eds) New Frontiers in Artificial Intelligence. JSAI-isAI 2021. Lecture Notes in Computer Science(), vol 13856. Springer, Cham. https://doi.org/10.1007/978-3-031-36190-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-36190-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36189-0
Online ISBN: 978-3-031-36190-6
eBook Packages: Computer ScienceComputer Science (R0)