Abstract
Focusing only on relevance when training recommender systems can lead to suboptimal recommendations. On eCommerce platforms, this not only leads to poor user experience for customers but also limits opportunities for vendors due to popularity bias. Consequently, there is an increasing interest in systems that produce recommendations that are not only relevant but also diverse. In this work, we have shown how diversity can be measured and improved in the context of a large eCommerce recommendation platform. We first evaluate how diversity should be measured feasibly in real-world recommender systems and introduce TILD: a novel diversity metric that measures representational diversity across user sessions. Second, we introduce VAE-based Generation of User Profiles (VAE-GUP), wherein we use the generative nature of VAEs to learn a user profile distribution and sample multiple user profiles. In a simple post-processing step, we combine the recommendations generated by these profiles to generate a final list of more diverse recommendations. Through empirical analysis of benchmark and real-world datasets, we show that VAE-GUP can improve several diversity metrics without significantly compromising relevance. This research has been done in collaboration with Bol.com, the largest e-commerce retailer in the Benelux region.
S. Borar—Work was done prior to joining Amazon.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adomavicius, G., Kwon, Y.: Improving aggregate recommendation diversity using ranking-based techniques. IEEE Trans. Knowl. Data Eng. 24(5), 896–911 (2012). https://doi.org/10.1109/TKDE.2011.15
Anderson, C.: The long tail (2004). https://www.wired.com/2004/10/tail/
Aytekin, T., Karakaya, M.Ö.: Clustering-based diversity improvement in top-n recommendation. J. Intell. Inf. Syst. 42(1), 1–18 (2014)
Bradley, K., Smyth, B.: Improving recommendation diversity (2001)
Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 335–336. SIGIR 1998, Association for Computing Machinery, New York, NY, USA (1998). https://doi.org/10.1145/290941.291025. https://doi-org.libproxy.aalto.fi/10.1145/290941.291025
Chaney, A.J.B., Stewart, B.M., Engelhardt, B.E.: How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In: Proceedings of the 12th ACM Conference on Recommender Systems. ACM (2018). https://doi.org/10.1145/3240323.3240370
Dacrema, M.F., Cremonesi, P., Jannach, D.: Are we really making much progress? a worrying analysis of recent neural recommendation approaches. In: Proceedings of the 13th ACM Conference on Recommender Systems. ACM (2019). https://doi.org/10.1145/3298689.3347058
Guo, W., Krauth, K., Jordan, M.I., Garg, N.: The stereotyping problem in collaboratively filtered recommender systems (2021)
Harper, F.M., Konstan, J.A.: The MovieLens datasets: History and context. ACM Trans. Interact. Intell. Syst. 5(4), 2827872 (2015). https://doi.org/10.1145/2827872
Jain, U., Zhang, Z., Schwing, A.G.: Creativity: generating diverse questions using variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst.(TOIS) 20(4), 422–446 (2002)
Kingma, D.P., Welling, M.: An introduction to variational autoencoders. Found. Trends® Mach. Learn. 12(4), 307–392 (2019). https://doi.org/10.1561/2200000056
Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adap. Inter. 22(4), 441–504 (2012)
Kunaver, M., Požrl, T.: Diversity in recommender systems-a survey. Knowl.-Based Syst. 123, 154–162 (2017)
Lathia, N., Hailes, S., Capra, L., Amatriain, X.: Temporal diversity in recommender systems. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 210–217. SIGIR 2010, Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1835449.1835486
Liang, D., Krishnan, R.G., Hoffman, M.D., Jebara, T.: Variational autoencoders for collaborative filtering (2018)
Liu, J.G., Shi, K., Guo, Q.: Solving the accuracy-diversity dilemma via directed random walks. Phys. Rev. E 85(1), 016118 (2012). https://doi.org/10.1103/physreve.85.016118
Mansoury, M., Abdollahpouri, H., Pechenizkiy, M., Mobasher, B., Burke, R.: Feedback loop and bias amplification in recommender systems. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2145–2148. CIKM 2020, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3340531.3412152. https://doi-org.libproxy.aalto.fi/10.1145/3340531.3412152
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2019). http://arxiv.org/abs/1908.10084
Wang, L., Schwing, A., Lazebnik, S.: Diverse and accurate image description using a variational auto-encoder with an additive gaussian encoding space. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper/2017/file/4b21cf96d4cf612f239a6c322b10c8fe-Paper.pdf
Wasilewski, J., Hurley, N.: Incorporating diversity in a learning to rank recommender system. In: the Twenty-ninth International Flairs Conference (2016)
Zhang, Y., Wang, Y., Zhang, L., Zhang, Z., Gai, K.: Improve diverse text generation by self labeling conditional variational auto encoder. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2767–2771 (2019). https://doi.org/10.1109/ICASSP.2019.8683090
Ziegler, C.N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: Proceedings of the 14th International Conference on World Wide Web, pp. 22–32. WWW 2005, Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1060745.1060754
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Appendix: Diversity Metric Comparison
A Appendix: Diversity Metric Comparison
Table 6 provides a comparison of all the diversity measures described in this paper.
k is the size of the recommendation list, U is the total number of users, and I is the total number of items. S is the set of items in the top K recommendations. \( I_K (\{v_j\}^n_j=1)\) is the space of all the subsets of I of size k.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Borar, S., Weerts, H., Gebre, B., Pechenizkiy, M. (2023). Improving Recommender System Diversity with Variational Autoencoders. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds) Advances in Bias and Fairness in Information Retrieval. BIAS 2023. Communications in Computer and Information Science, vol 1840. Springer, Cham. https://doi.org/10.1007/978-3-031-37249-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-37249-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37248-3
Online ISBN: 978-3-031-37249-0
eBook Packages: Computer ScienceComputer Science (R0)