skip to main content
10.1145/3488560.3498477acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Variational User Modeling with Slow and Fast Features

Published: 15 February 2022 Publication History

Abstract

Recommender systems play a key role in helping users find their favorite music to play among an often extremely large catalog of items on online streaming services. To correctly identify users' interests, recommendation algorithms rely on past user behavior and feedback to aim at learning users' preferences through the logged interactions. User modeling is a fundamental part of this large-scale system as it enables the model to learn an optimal representation for each user. For instance, in music recommendation, the focus of this paper, users' interests at any time is shaped by their general preferences for music as well as their recent or momentary interests in a particular type of music. In this paper, we present a novel approach for learning user representation based on general and slow-changing user interests as well as fast-moving current preferences. We propose a variational autoencoder-based model that takes fast and slow-moving features and learns an optimal user representation. Our model, which we call FS-VAE, consists of sequential and non-sequential encoders to capture patterns in user-item interactions and learn users' representations. We evaluate FS-VAE on a real-world music streaming dataset. Our experimental results show a clear improvement in learning optimal representations compared to state-of-the-art baselines on the next item recommendation task. We also demonstrate how each of the model components, slow input feature, and fast ones play a role in achieving the best results in next item prediction and learning users' representations.

Supplementary Material

MP4 File (WSDM22-fp539.mp4)
This video provides a 10-minute presentation of our paper titled ?Variational User Modeling with Slow and Fast Features?. In this work, we propose a novel approach for learning users' representations in recommender systems. We develop fast and slow-moving features to learn general and momentary user interests with a focus on music streaming applications. The variational autoencoder-based model processes sequential and non-sequential features with recurrent and regular components. We optimize the model over a next-item prediction task. Our experimental results show clear improvement over the current baselines. We investigate various components of the model and highlight the importance of each part through an ablation study.

References

[1]
Charu C Aggarwal. 2016. Content-based recommender systems. In Recommender systems . Springer, 139--166.
[2]
Xavier Amatriain and Justin Basilico. 2016. Past, present, and future of recommender systems: An industry perspective. In Proceedings of the 10th ACM Conference on Recommender Systems. 211--214.
[3]
Zeynep Batmaz, Ali Yurekli, Alper Bilge, and Cihan Kaleli. 2019. A review on deep learning for recommender systems: challenges and remedies. Artificial Intelligence Review, Vol. 52, 1 (2019), 1--37.
[4]
Robert M Bell and Yehuda Koren. 2007. Lessons from the Netflix prize challenge. Acm Sigkdd Explorations Newsletter, Vol. 9, 2 (2007), 75--79.
[5]
Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H Chi. 2018. Latent cross: Making use of context in recurrent recommender systems. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining . 46--54.
[6]
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. 2017. Variational inference: A review for statisticians. Journal of the American statistical Association, Vol. 112, 518 (2017), 859--877.
[7]
Geoffray Bonnin and Dietmar Jannach. 2014. Automated generation of music playlists: Survey and experiments. ACM Computing Surveys (CSUR), Vol. 47, 2 (2014), 1--35.
[8]
Brian Brost, Rishabh Mehrotra, and Tristan Jehan. 2019. The music streaming sessions dataset. In The World Wide Web Conference . 2594--2600.
[9]
Erion cC ano and Maurizio Morisio. 2017. Hybrid recommender systems: A systematic literature review. Intelligent Data Analysis, Vol. 21, 6 (2017), 1487--1524.
[10]
Toni Cebrián, Marc Planagumà, Paulo Villegas, and Xavier Amatriain. 2010. Music recommendations with temporal context awareness. In Proceedings of the fourth ACM conference on Recommender systems. 349--352.
[11]
Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Le Song. 2016. Deep coevolutionary network: Embedding user and item features for recommendation. arXiv preprint arXiv:1609.03675 (2016).
[12]
Hui Fang, Danning Zhang, Yiheng Shu, and Guibing Guo. 2020. Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations. ACM Transactions on Information Systems (TOIS), Vol. 39, 1 (2020), 1--42.
[13]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning .MIT press.
[14]
Prem Gopalan, Jake M Hofman, and David M Blei. 2015. Scalable Recommendation with Hierarchical Poisson Factorization. In UAI . 326--335.
[15]
Prem Gopalan, Francisco J Ruiz, Rajesh Ranganath, and David Blei. 2014. Bayesian nonparametric poisson factorization for recommendation systems. In Artificial Intelligence and Statistics. PMLR, 275--283.
[16]
Casper Hansen, Christian Hansen, Lucas Maystre, Rishabh Mehrotra, Brian Brost, Federico Tomasi, and Mounia Lalmas. 2020. Contextual and sequential user embeddings for large-scale music recommendation. In Fourteenth ACM Conference on Recommender Systems. 53--62.
[17]
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-vae: Learning basic visual concepts with a constrained variational framework. (2016).
[18]
Matthew D Hoffman and Matthew J Johnson. 2016. Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS, Vol. 1.
[19]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining. Ieee, 263--272.
[20]
Folasade Olubusola Isinkaye, YO Folajimi, and Bolande Adefowoke Ojokoh. 2015. Recommendation systems: Principles, methods and evaluation. Egyptian informatics journal, Vol. 16, 3 (2015), 261--273.
[21]
Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
[22]
Yehuda Koren. 2009. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining . 447--456.
[23]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.
[24]
Pigi Kouki, Shobeir Fakhraei, James Foulds, Magdalini Eirinaki, and Lise Getoor. 2015. Hyper: A flexible and extensible probabilistic framework for hybrid recommender systems. In Proceedings of the 9th ACM Conference on Recommender Systems. 99--106.
[25]
Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 1269--1278.
[26]
Huoran Li, Xuan Lu, Xuanzhe Liu, Tao Xie, Kaigui Bian, Felix Xiaozhu Lin, Qiaozhu Mei, and Feng Feng. 2015. Characterizing smartphone usage patterns from millions of android users. In Proceedings of the 2015 Internet Measurement Conference. 459--472.
[27]
Sheng Li and Handong Zhao. 2020. A Survey on Representation Learning for User Modeling. In IJCAI . 4997--5003.
[28]
Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference. 689--698.
[29]
Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz. 2018. Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In Proceedings of the 27th acm international conference on information and knowledge management . 2243--2251.
[30]
Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems. 1257--1264.
[31]
Aanchal Mongia, Neha Jhamb, Emilie Chouzenoux, and Angshul Majumdar. 2020. Deep latent factor model for collaborative filtering. Signal Processing, Vol. 169 (2020), 107366.
[32]
James Neve and Ivan Palomares. 2019. Latent factor models and aggregation operators for collaborative filtering in reciprocal recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems. 219--227.
[33]
Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequence-aware recommender systems. ACM Computing Surveys (CSUR), Vol. 51, 4 (2018), 1--36.
[34]
Shaina Raza and Chen Ding. 2019. Progress in context-aware recommender systems-An overview. Computer Science Review, Vol. 31 (2019), 84--97.
[35]
Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2019. Repeatnet: A repeat aware neural recommendation machine for session-based recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4806--4813.
[36]
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning . PMLR, 1278--1286.
[37]
Christian Robert and George Casella. 2013. Monte Carlo statistical methods .Springer Science & Business Media.
[38]
Noveen Sachdeva, Giuseppe Manco, Ettore Ritacco, and Vikram Pudi. 2019. Sequential variational autoencoders for collaborative filtering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining . 600--608.
[39]
Markus Schedl. 2019. Deep learning in music recommendation systems. Frontiers in Applied Mathematics and Statistics, Vol. 5 (2019), 44.
[40]
Markus Schedl, Hamed Zamani, Ching-Wei Chen, Yashar Deldjoo, and Mehdi Elahi. 2018. Current challenges and visions in music recommender systems research. International Journal of Multimedia Information Retrieval, Vol. 7, 2 (2018), 95--116.
[41]
Andreu Vall, Massimo Quadrana, Markus Schedl, Gerhard Widmer, and Paolo Cremonesi. 2017. The Importance of Song Context in Music Playlists. In RecSys Posters .
[42]
Steven Van Canneyt, Marc Bron, Andy Haines, and Mounia Lalmas. 2017. Describing patterns and disruptions in large scale mobile app usage data. In Proceedings of the 26th International Conference on World Wide Web Companion . 1579--1584.
[43]
Dongjing Wang, Shuiguang Deng, and Guandong Xu. 2018. Sequence-based context-aware music recommendation. Information Retrieval Journal, Vol. 21, 2 (2018), 230--252.
[44]
Shoujin Wang, Longbing Cao, Yan Wang, Quan Z Sheng, Mehmet A Orgun, and Defu Lian. 2021. A survey on session-based recommender systems. ACM Computing Surveys (CSUR), Vol. 54, 7 (2021), 1--38.
[45]
Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. 2019. Sequential recommender systems: challenges, progress and prospects. arXiv preprint arXiv:2001.04830 (2019).
[46]
Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining. 495--503.
[47]
Di Wu, Xin Luo, Mingsheng Shang, Yi He, Guoyin Wang, and MengChu Zhou. 2019. A deep latent factor model for high-dimensional and sparse matrices in recommender systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2019).
[48]
Teng Xiao and Hong Shen. 2019. Neural variational matrix factorization for collaborative filtering in recommendation systems. Applied Intelligence, Vol. 49, 10 (2019), 3558--3569.
[49]
Yingyuan Xiao, Pengqiang Ai, Ching-Hsien Hsu, Hongya Wang, and Xu Jiao. 2015. Time-ordered collaborative filtering for news recommendation. China Communications, Vol. 12, 12 (2015), 53--62.
[50]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), Vol. 52, 1 (2019), 1--38.
[51]
Fan Zhou, Zijing Wen, Kunpeng Zhang, Goce Trajcevski, and Ting Zhong. 2019. Variational session-based recommendation using normalizing flows. In The World Wide Web Conference. 3476--3475.

Cited By

View all
  • (2024)Dynamic Relation Graph Learning for Time-Aware Service RecommendationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.332597721:2(1503-1517)Online publication date: Apr-2024
  • (2024)Integrating Repeat Listening Patterns for Enhanced Music Recommendation2024 18th International Conference on Ubiquitous Information Management and Communication (IMCOM)10.1109/IMCOM60618.2024.10418302(1-7)Online publication date: 3-Jan-2024
  • (2024)Mobile Applications in Smart Tourism: Implementing User ModellingSmart Tourism–The Impact of Artificial Intelligence and Blockchain10.1007/978-3-031-50883-7_4(53-74)Online publication date: 2-Feb-2024
  • Show More Cited By

Index Terms

  1. Variational User Modeling with Slow and Fast Features

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
    February 2022
    1690 pages
    ISBN:9781450391320
    DOI:10.1145/3488560
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 February 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. latent representation
    2. music streaming
    3. user modeling

    Qualifiers

    • Research-article

    Conference

    WSDM '22

    Acceptance Rates

    Overall Acceptance Rate 498 of 2,863 submissions, 17%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)71
    • Downloads (Last 6 weeks)15
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Dynamic Relation Graph Learning for Time-Aware Service RecommendationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.332597721:2(1503-1517)Online publication date: Apr-2024
    • (2024)Integrating Repeat Listening Patterns for Enhanced Music Recommendation2024 18th International Conference on Ubiquitous Information Management and Communication (IMCOM)10.1109/IMCOM60618.2024.10418302(1-7)Online publication date: 3-Jan-2024
    • (2024)Mobile Applications in Smart Tourism: Implementing User ModellingSmart Tourism–The Impact of Artificial Intelligence and Blockchain10.1007/978-3-031-50883-7_4(53-74)Online publication date: 2-Feb-2024
    • (2023)MUSE: Music Recommender System with Shuffle Play Recommendation EnhancementProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614976(1928-1938)Online publication date: 21-Oct-2023
    • (2023)Social Network Data Enabling Smart Tourism2023 14th International Conference on Information, Intelligence, Systems & Applications (IISA)10.1109/IISA59645.2023.10345898(1-6)Online publication date: 10-Jul-2023
    • (2022)Rethinking Personalized Ranking at Pinterest: An End-to-End ApproachProceedings of the 16th ACM Conference on Recommender Systems10.1145/3523227.3547394(502-505)Online publication date: 12-Sep-2022
    • (2022)Time-aware Service Recommendation with Social-powered Graph Hierarchical Attention NetworkIEEE Transactions on Services Computing10.1109/TSC.2022.3197655(1-12)Online publication date: 2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media