Skip to main content
SpringerLink
Log in

AudioLens: Audio-Aware Video Recommendation for Mitigating New Item Problem

  • Conference paper
  • First Online:
Service-Oriented Computing – ICSOC 2020 Workshops (ICSOC 2020)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12632))

Included in the following conference series:

Abstract

From the early years, the research on recommender systems has been largely focused on developing advanced recommender algorithms. These sophisticated algorithms are capable of exploiting a wide range of data, associated with video items, and build quality recommendations for users. It is true that the excellency of recommender systems can be very much boosted with the performance of their recommender algorithms. However, the most advanced algorithms may still fail to recommend video items that the system has no form of representative data associated to them (e.g., tags and ratings). This is a situation called New Item problem and it is part of a major challenge called Cold Start. This problem happens when a new item is added to the catalog of the system and no data is available for that item. This can be a serious issue in video-sharing applications where hundreds of hours of videos are uploaded in every minute, and considerable number of these videos may have no or very limited amount of associated data.

In this paper, we address this problem by proposing recommendation based on novel features that do not require human-annotation, as they can be extracted completely automatic. This enables these features to be used in the cold start situation where any other source of data could be missing. Our proposed features describe audio aspects of video items (e.g., energy, tempo, and danceability, and speechiness) which can capture a different (still important) picture of user preferences. While recommendation based on such preferences could be important, very limited attention has been paid to this type of approaches.

We have collected a large dataset of unique audio features (from Spotify) extracted from more than 9000 movies. We have conducted a set of experiments using this dataset and evaluated our proposed recommendation technique in terms of different metrics, i.e., Precision@K, Recall@K, RMSE, and Coverage. The results have shown the superior performance of recommendations based on audio features, used individually or combined, in the cold start evaluation scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.omnicoreagency.com/youtube-statistics.

  2. 2.

    http://tubularinsights.com/hours-minute-uploaded-youtube/.

  3. 3.

    https://github.com/mhrimaz/audio-lens.

  4. 4.

    https://developer.spotify.com/documentation/web-api/.

  5. 5.

    https://grouplens.org/datasets/movielens/25m/.

  6. 6.

    https://developer.spotify.com/web-api/get-audio-features.

  7. 7.

    https://en.wikipedia.org/wiki/Pitch_class.

  8. 8.

    https://zenodo.org/record/3266236#.Xx7hLPgzako.

  9. 9.

    https://www.filmindependent.org/blog/know-score-brief-history-film-music/.

References

  1. Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005). https://doi.org/10.1109/TKDE.2005.99

    Article  Google Scholar 

  2. Aggarwal, C.C.: Content-based recommender systems. In: Aggarwal, C.C. (ed.) Recommender Systems, pp. 139–166. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29659-3_4

  3. Anderson, C.: The Long Tail. Random House Business, New York (2006)

    Google Scholar 

  4. Bakhshandegan Moghaddam, F., Elahi, M.: Cold start solutions for recommendation systems. Big Data Recommender Systems, Recent Trends and Advances IET (2019)

    Google Scholar 

  5. Brezeale, D., Cook, D.J.: Automatic video classification: a survey of the literature. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 38(3), 416–430 (2008)

    Article  Google Scholar 

  6. Cantador, I., Bellogín, A., Vallet, D.: Content-based recommendation in social tagging systems. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 237–240. ACM (2010)

    Google Scholar 

  7. Cantador, I., Konstas, I., Jose, J.M.: Categorising social tags to improve folksonomy-based recommendations. Web Semant. Sci. Serv. Agents World Wide Web 9(1), 1–15 (2011)

    Article  Google Scholar 

  8. Cremonesi, P., Elahi, M., Garzotto, F.: User interface patterns in recommendation-empowered content intensive multimedia applications. Multimedia Tools Appl. 76(4), 5275–5309 (2016). https://doi.org/10.1007/s11042-016-3946-5

    Article  Google Scholar 

  9. Cremonesi, P., Koren, Y., Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 39–46 (2010)

    Google Scholar 

  10. De Gemmis, M., Lops, P., Semeraro, G., Basile, P.: Integrating tags in a semantic content-based recommender. In: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 163–170. ACM (2008)

    Google Scholar 

  11. Deldjoo, Y., Constantin, M.G., Eghbal-Zadeh, H., Ionescu, B., Schedl, M., Cremonesi, P.: Audio-visual encoding of multimedia content for enhancing movie recommendations. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, New York, NY, USA, pp. 455–459. Association for Computing Machinery (2018). https://doi.org/10.1145/3240323.3240407

  12. Deldjoo, Y., Elahi, M., Cremonesi, P., Garzotto, F., Piazzolla, P., Quadrana, M.: Content-based video recommendation system based on stylistic visual features. J. Data Semant., 1–15 (2016)

    Google Scholar 

  13. Di Noia, T., Mirizzi, R., Ostuni, V.C., Romito, D., Zanker, M.: Linked open data to support content-based recommender systems. In: Proceedings of the 8th International Conference on Semantic Systems, pp. 1–8. ACM (2012)

    Google Scholar 

  14. Elahi, M.: Empirical evaluation of active learning strategies in collaborative filtering. Ph.D. thesis, Ph.D. Dissertation. Free University of Bozen-Bolzano (2014)

    Google Scholar 

  15. Elahi, M., Braunhofer, M., Gurbanov, T., Ricci, F.: User preference elicitation, rating sparsity and cold start (2018)

    Google Scholar 

  16. Elahi, M., Hosseini, R., Rimaz, M.H., Moghaddam, F.B., Trattner, C.: Visually-aware video recommendation in the cold start. In: Proceedings of the 31st ACM Conference on Hypertext and Social Media, pp. 225–229 (2020)

    Google Scholar 

  17. Elahi, M., Ricci, F., Rubens, N.: A survey of active learning in collaborative filtering recommender systems. Comput. Sci. Rev. 20, 29–50 (2016)

    Article  MathSciNet  Google Scholar 

  18. Enrich, M., Braunhofer, M., Ricci, F.: Cold-start management with cross-domain collaborative filtering and tags. In: Huemer, C., Lops, P. (eds.) EC-Web 2013. LNBIP, vol. 152, pp. 101–112. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39878-0_10

    Chapter  Google Scholar 

  19. Ercegovac, I.R., Dobrota, S., Kuščević, D.: Relationship between music and visual art preferences and some personality traits. Empirical Stud. Arts 33(2), 207–227 (2015). https://doi.org/10.1177/0276237415597390

  20. Gedikli, F., Jannach, D.: Improving recommendation accuracy based on item-specific tag preferences. ACM Trans. Intell. Sys. Technol. (TIST) 4(1), 11 (2013)

    Google Scholar 

  21. Gillick, J., Bamman, D.: Telling stories with soundtracks: an empirical analysis of music in film. In: Proceedings of the First Workshop on Storytelling, New Orleans, Louisiana, pp. 33–42. Association for Computational Linguistics, June 2018. https://doi.org/10.18653/v1/W18-1504. https://www.aclweb.org/anthology/W18-1504

  22. Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4) (2015). https://doi.org/10.1145/2827872

  23. Hazrati, N., Elahi, M.: Addressing the new item problem in video recommender systems by incorporation of visual features with restricted Boltzmann machines. Expert Syst. 38, e12645 (2020)

    Google Scholar 

  24. Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5–53 (2004). https://doi.org/10.1145/963770.963772

    Article  Google Scholar 

  25. Hornick, M.F., Tamayo, P.: Extending recommender systems for disjoint user/item sets: the conference recommendation problem. IEEE Trans. Knowl. Data Eng. 8, 1478–1490 (2012)

    Article  Google Scholar 

  26. Hu, W., Xie, N., Li, Zeng, X., Maybank, S.: A survey on visual content-based video indexing and retrieval. Trans. Sys. Man Cyber Part C 41(6), 797–819 (2011). https://doi.org/10.1109/TSMCC.2011.2109710

  27. Jannach, D., Zanker, M., Felfernig, A., Friedrich, G.: Recommender Systems: An Introduction. Cambridge University Press, Cambridge (2010)

    Book  Google Scholar 

  28. Liang, H., Xu, Y., Li, Y., Nayak, R.: Tag based collaborative filtering for recommender systems. In: Wen, P., Li, Y., Polkowski, L., Yao, Y., Tsumoto, S., Wang, G. (eds.) RSKT 2009. LNCS (LNAI), vol. 5589, pp. 666–673. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02962-2_84

    Chapter  Google Scholar 

  29. Lika, B., Kolomvatsos, K., Hadjiefthymiades, S.: Facing the cold start problem in recommender systems. Expert Syst. Appl. 41(4), 2065–2073 (2014)

    Article  Google Scholar 

  30. Lops, P., de Gemmis, M., Semeraro, G.: Content-based recommender systems: state of the art and trends. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 73–105. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_3

    Chapter  Google Scholar 

  31. Melchiorre, A.B., Schedl, M.: Personality correlates of music audio preferences for modelling music listeners. In: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, New York, NY, USA, pp. 313–317. Association for Computing Machinery (2020). https://doi.org/10.1145/3340631.3394874

  32. Milicevic, A.K., Nanopoulos, A., Ivanovic, M.: Social tagging in recommender systems: a survey of the state-of-the-art and possible extensions. Artif. Intell. Rev. 33(3), 187–209 (2010)

    Article  Google Scholar 

  33. Resnick, P., Varian, H.R.: Recommender systems. Commun. ACM 40(3), 56–58 (1997). https://doi.org/10.1145/245108.245121

    Article  Google Scholar 

  34. Ricci, F., Rokach, L., Shapira, B.: Recommender systems: introduction and challenges. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 1–34. Springer, Boston, MA (2015). https://doi.org/10.1007/978-1-4899-7637-6_1

    Chapter  MATH  Google Scholar 

  35. Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.): Recommender Systems Handbook. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3

    Book  MATH  Google Scholar 

  36. Rimaz, M.H., Elahi, M., Bakhshandegan Moghadam, F., Trattner, C., Hosseini, R., Tkalčič, M.: Exploring the power of visual features for the recommendation of movies. In: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization, pp. 303–308 (2019)

    Google Scholar 

  37. Rubens, N., Elahi, M., Sugiyama, M., Kaplan, D.: Active learning in recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 809–846. Springer, Boston (2015). https://doi.org/10.1007/978-1-4899-7637-6_24

    Chapter  Google Scholar 

  38. Schedl, M., Zamani, H., Chen, C.-W., Deldjoo, Y., Elahi, M.: Current challenges and visions in music recommender systems research. Int. J. Multimed. Inf. Retr. 7(2), 95–116 (2018). https://doi.org/10.1007/s13735-018-0154-2

    Article  Google Scholar 

  39. Shepitsen, A., Gemmell, J., Mobasher, B., Burke, R.: Personalized recommendation in social tagging systems using hierarchical clustering. In: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 259–266. ACM (2008)

    Google Scholar 

  40. Tkalčič, M., Maleki, N., Pesek, M., Elahi, M., Ricci, F., Marolt, M.: A research tool for user preferences elicitation with facial expressions. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 353–354. ACM (2017)

    Google Scholar 

  41. Tkalčič, M., Maleki, N., Pesek, M., Elahi, M., Ricci, F., Marolt, M.: Prediction of music pairwise preferences from facial expressions. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, IUI 2019, New York, NY, USA, pp. 150–159. Association for Computing Machinery (2019). https://doi.org/10.1145/3301275.3302266

  42. Vlachos, M., Duenner, C., Heckel, R., Vassiliadis, V.G., Parnell, T., Atasu, K.: Addressing interpretability and cold-start in matrix factorization for recommender systems. IEEE Trans. Knowl. Data Eng. 31, 1253–1266 (2018)

    Article  Google Scholar 

  43. Wang, L., Zeng, X., Koehl, L., Chen, Y.: Intelligent fashion recommender system: fuzzy logic in personalized garment design. IEEE Trans. Hum.-Mach. Syst. 45(1), 95–109 (2015)

    Article  Google Scholar 

  44. Xu, H., Goonawardene, N.: Does movie soundtrack matter? The role of soundtrack in predicting movie revenue. In: Siau, K., Li, Q., Guo, X. (eds.) 18th Pacific Asia Conference on Information Systems, PACIS 2014, Chengdu, China, 24–28 June 2014, p. 350 (2014). http://aisel.aisnet.org/pacis2014/350

Download references

Author information

Authors and Affiliations

  1. Technical University of Kaiserslautern, Erwin-Schrödinger-Str 52, 67663, Kaiserslautern, Germany

    Mohammad Hossein Rimaz

  2. Vaillant Group Business Services, Berghauser Str. 63, 42859, Remscheid, Germany

    Reza Hosseini

  3. University of Bergen, Fosswinckelsgt. 6, 5007, Bergen, Norway

    Mehdi Elahi

  4. University of Bonn, Regina-Pacis-Weg 3, 53113, Bonn, Germany

    Farshad Bakhshandegan Moghaddam

Authors
  1. Mohammad Hossein Rimaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehdi Elahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farshad Bakhshandegan Moghaddam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Hossein Rimaz .

Editor information

Editors and Affiliations

  1. Zayed University, Dubai, United Arab Emirates

    Hakim Hacid

  2. Zayed University, Dubai, United Arab Emirates

    Fatma Outay

  3. University of New South Wales, Sydney, NSW, Australia

    Hye-young Paik

  4. Huawei Paris Research Center, Paris, France

    Amira Alloum

  5. National Research Council C.N.R., Pisa, Italy

    Marinella Petrocchi

  6. Deakin University, Waurn Ponds, VIC, Australia

    Mohamed Reda Bouadjenek

  7. Macquarie University, Sydney, NSW, Australia

    Amin Beheshti

  8. Rochester Institute of Technology, Rochester, NY, USA

    Xumin Liu

  9. Université de Paris, Paris, France

    Abderrahmane Maaradji

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rimaz, M.H., Hosseini, R., Elahi, M., Moghaddam, F.B. (2021). AudioLens: Audio-Aware Video Recommendation for Mitigating New Item Problem. In: Hacid, H., et al. Service-Oriented Computing – ICSOC 2020 Workshops. ICSOC 2020. Lecture Notes in Computer Science(), vol 12632. Springer, Cham. https://doi.org/10.1007/978-3-030-76352-7_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76352-7_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76351-0

  • Online ISBN: 978-3-030-76352-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation