Iman Kamehkhosh
Dietmar Jannach
ABSTRACT
Many of today's music streaming websites and apps provide personalized next-track listening recommendations based on the user's current and past listening behavior. In the research literature, various algorithmic approaches to determine suitable next tracks can be found. However, almost all of them were evaluated in offline experiments using, for example, manually created playlists as a gold standard. In this work, we aim to check the external validity of insights that are obtained through such offline experiments on historical datasets. We conducted an online user study involving 277 subjects in which the participants evaluated the suitability of four different alternatives of continuing a given set of playlists. Our results indicate that manually created playlists can in fact represent a reasonable gold standard, an insight for which no evidence existed in the literature before. Furthermore, our work was able to confirm that considering playlist homogeneity aspects does not only lead to performance improvements in offline experiments -- as indicated by past research -- but also to a better quality perception by users. However, the observations also revealed that user studies of this type can be easily distorted by item familiarity biases, because the participants tend to evaluate continuation alternatives better when they know the track or the artist.
- Ivana Andjelkovic, Denis Parra, and John O'Donovan. 2016. Moodplay: Interactive Mood-based Music Discovery and Recommendation. In UMAP '16. 275--279. Google Scholar
Digital Library
- Luke Barrington, Reid Oda, and Gert R. G. Lanckriet. 2009. Smarter than Genius? Human Evaluation of Music Recommender Systems. In ISMIR '09. 357--362.Google Scholar
- Geoffray Bonnin and Dietmar Jannach. 2013. Evaluating The Quality of Playlists Based on Hand-Crafted Samples. In ISMIR '13. 263--268.Google Scholar
- Geoffray Bonnin and Dietmar Jannach. 2014. Automated Generation of Music Playlists: Survey and Experiments. ACM Computing Surveys 47, 2 (2014), 26:1-- 26:35. Google ScholarDigital Library
- Matthias Braunhofer, Marius Kaminskas, and Francesco Ricci. 2013. Locationaware Music Recommendation. International Journal of Multimedia Information Retrieval 2, 1 (2013), 31--44.Google ScholarCross Ref
- Pei-I Chen, Jen-Yu Liu, and Yi-Hsuan Yang. 2015. Personal Factors in Music Preference and Similarity: User Study on The Role of Personality Traits. In CMMR '15.Google Scholar
- Susan Craw, Ben Horsburgh, and Stewart Massie. Music Recommenders: User Evaluation Without Real Users'. In IJCAI '15. 1749--1755. Google ScholarDigital Library
- Paolo Cremonesi, Franca Garzotto, Sara Negro, Alessandro Papadopoulos, and Roberto Turrin. 2011. Comparative Evaluation of Recommender System Quality. In CHI EA'11. 1927--1932. Google ScholarDigital Library
- Paolo Cremonesi, Franca Garzotto, and Roberto Turrin. 2012. Investigating The Persuasion Potential of Recommender Systems from a Quality Perspective: An Empirical Study. Transactions on Interactive Intelligent Systems 2, 2 (2012), 11:1--11:41. Google ScholarDigital Library
- Sally Jo. Cunningham, David. Bainbridge, and Annette. Falconer. 2006. 'More of an Art than a Science': Supporting The Creation of Playlists and Mixes. In ISMIR '06. 240--245.Google Scholar
- Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User Perception of Differences in Recommender Algorithms. In RecSys '14. 161--168. Google ScholarDigital Library
- Peter Emerson. 2013. The Original Borda Count and Partial Voting. Social Choice and Welfare 40, 2 (2013), 353--358.Google ScholarCross Ref
- Bruce Ferwerda, Mark Graus, Andreu Vall, Marko Tkalčič, and Markus Schedl. 2016. The Influence of Users' Personality Traits on Satisfaction and Attractiveness of Diversified Recommendation Lists. In EMPIRE '16 Workshop at RecSys '16. 43-- 47.Google Scholar
- Florent Garcin, Boi Faltings, Olivier Donatsch, Ayar Alazzawi, Christophe Bruttin, and Amr Huber. 2014. Offline and Online Evaluation of News Recommender Systems at swissinfo.ch. In RecSys '14. 169--176. Google ScholarDigital Library
- Carlos A. Gomez-Uribe and Neil Hunt. 2015. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Transactions on Management Information Systems 6, 4 (2015), 13:1--13:19. Google ScholarDigital Library
- Derek L. Hansen and Jennifer Golbeck. 2009. Mixing It Up: Recommending Collections of Items. In CHI '09. 1217--1226. Google ScholarDigital Library
- Negar Hariri, Bamshad Mobasher, and Robin Burke. 2012. Context-Aware Music Recommendation Based on Latent Topic Sequential Paterns. In RecSys '12. 131-- 138. Google ScholarDigital Library
- Xiao Hu, J. Stephen Downie, and Andreas F. Ehmann. 2009. Lyric Text Mining in Music Mood Classification. In ISMIR '09. 411--416.Google Scholar
- Dietmar Jannach and Kolja Hegelich. 2009. A Case Study on The Effectiveness of Recommendations in The Mobile Internet. In RecSys '09. 205--208. Google ScholarDigital Library
- Dietmar Jannach, Iman Kamehkhosh, and Geoffray Bonnin. 2014. Analyzing The Characteristics of Shared Playlists for Music Recommendation. In RSWeb '14 Workhop at RecSys '14.Google Scholar
- Dietmar Jannach, Iman Kamehkhosh, and Geoffray Bonnin. 2016. Biases in Automated Music Playlist Generation: A Comparison of Next-Track Recommending Techniques. In UMAP '16. 281--285. Google ScholarDigital Library
- Dietmar Jannach, Iman Kamehkhosh, and Lukas Lerche. 2017. Leveraging MultiDimensional User Models for Personalized Next-Track Music Recommendation. In SAC '17. Google ScholarDigital Library
- Dietmar Jannach, Lukas Lerche, and Michael Jugovac. 2015. Item familiarity as a possible confounding factor in user-centric recommender systems evaluation. i-com Journal for Interactive Media 14, 1 (2015), 29--40.Google Scholar
- Dietmar Jannach, Lukas Lerche, and Iman Kamehkhosh. 2015. Beyond "Hitting The Hits": Generating Coherent Music Playlist Continuations with The Right Tracks. In RecSys '15. 187--194. Google ScholarDigital Library
- Dietmar Jannach, Markus Zanker, Mouzhi Ge, and Marian Gröning. 2012. Rec-ommender Systems in Computer Science and Information Systems--A Landscape of Research. In EC-Web '12. 76--87.Google Scholar
- Evan Kirshenbaum, George Forman, and Michael Dugan. 2012. A Live Comparison of Methods for Personalized Article Recommendation at Forbes.com. In ECML/PKDD '12. 51--66.Google ScholarCross Ref
- Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining The User Experience of Recommender Systems. User Modeling and User-Adapted Interaction 4--5 (2012), 441--504. Google ScholarDigital Library
- Jin Ha Lee and Rachel Price. 2016. User experience with commercial music services: An empirical exploration. Journal of The Association for Information Science and Technology 67, 4 (2016), 800--811. Google ScholarDigital Library
- Brian McFee and Gert R. G. Lanckriet. 2011. The Natural Language of Playlists. In ISMIR '11. 537--542.Google Scholar
- Brian McFee and Gert R. G. Lanckriet. 2012. Hypergraph Models of Playlist Dialects. In ISMIR '12. 343--348.Google Scholar
- Joshua L Moore, Shuo Chen, Thorsten Joachims, and Douglas Turnbull. 2012. Learning to Embed Songs and Tags for Playlist Prediction. In ISMIR '12. 349--354.Google Scholar
- Melissa Onori, Alessandro Micarelli, and Giuseppe Sansonetti. 2016. A Comparative Analysis of Personality-Based Music Recommender Systems. In EMPIRE '16 Workshop at RecSys '16. 55--59.Google Scholar
- John C. Platt, Christopher J. C. Burges, Steven Swenson, Christopher Weare, and Alice Zheng. 2001. Learning a Gaussian Process Prior for Automatically Generating Music Playlists. In NIPS '11. 1425--1432. Google ScholarDigital Library
- George Popescu and Pearl Pu. 2012. What's The Best Music You Have?: Designing Music Recommendation for Group Enjoyment in Groupfun. In CHI EA '12. 1673-- 1678. Google ScholarDigital Library
- Pearl Pu, Li Chen, and Rong Hu. 2011. A User-centric Evaluation Framework for Recommender Systems. In RecSys '11. 157--164. Google ScholarDigital Library
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI '09. 452--461. Google ScholarDigital Library
- Marco Rossetti, Fabio Stella, and Markus Zanker. 2016. Contrasting Offline and Online Results when Evaluating Recommendation Algorithms. In RecSys '16. 31--34. Google ScholarDigital Library
- Andy M. Sarroff and Michael Casey. 2012. Modeling and Predicting Song Adjacencies In Commercial Albums. In SMC '12.Google Scholar
- Malcolm Slaney and William White. 2006. Measuring Playlist Diversity for Recommendation Systems. In AMCMM '06. 77--82. Google ScholarDigital Library
- Roberto Turrin, Massimo Quadrana, Andrea Condorelli, Roberto Pagano, and Paolo Cremonesi. 2015. 30Music Listening and Playlists Dataset. In Poster Proceedings RecSys '15.Google Scholar
- Linxing Xiao, Lie Lu, Frank Seide, and Jie Zhou. 2009. Learning a Music Similarity Measure on Automatic Annotations with Application to Playlist Generation. In ICASSP '09. 1885--1888. Google ScholarDigital Library
Index Terms
- User Perception of Next-Track Music Recommendations
Recommendations
Tailoring Music Recommendations to Users by Considering Diversity, Mainstreaminess, and Novelty
SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information RetrievalA shortcoming of current approaches for music recommendation is that they consider user-specific characteristics only on a very simple level, typically as some kind of interaction between users and items when employing collaborative filtering. To ...
Learning to embed music and metadata for context-aware music recommendation
Contextual factors greatly influence users' musical preferences, so they are beneficial remarkably to music recommendation and retrieval tasks. However, it still needs to be studied how to obtain and utilize the contextual information. In this paper, we ...
A music recommendation system based on music and user grouping
Special issue: Intelligent multimedia applicationsIn this paper, we present a music recommendation system, which provides a personalized service of music recommendation. The polyphonic music objects of MIDI format are first analyzed for deriving information for music grouping. For this purpose, the ...
Comments