research-article

Visually-Aware Video Recommendation in the Cold Start

Authors:

Mohammad H. Rimaz,

Farshad B. Moghaddam,

Christoph TrattnerAuthors Info & Claims

HT '20: Proceedings of the 31st ACM Conference on Hypertext and Social Media

Pages 225 - 229

https://doi.org/10.1145/3372923.3404778

Published: 13 July 2020 Publication History

Abstract

Recommender Systems have become essential tools in any modern video-sharing platform. Although, recommender systems have shown to be effective in generating personalized suggestions in video-sharing platforms, however, they suffer from the so-called New Item problem. New item problem, as part of Cold Start problem, happens when a new item is added to the system catalogue and the recommender system has no or little data available for that new item. In such a case, the system may fail to meaningfully recommend the new item to the users.

In this paper, we propose a novel recommender system that is based on visual tags, i.e., tags that are automatically annotated to videos based on visual description of the videos. Such visual tags can be used in an extreme cold start situation, where neither any rating, nor any tag is available for the new video. The visual tags could also be used in the moderate cold start situation when the new video might have been annotated with few tags. This type of content features can be extracted automatically without any human involvement and have been shown to be very effective in representing the video content.

We have used a large dataset of videos and shown that automatically extracted visual tags can be incorporated into the cold start recommendation process and achieve superior results compared to the recommendation based on human-annotated tags.

References

[1]

Toine Bogers. 2018. Tag-based recommendation. In Social Information Access. Springer, 441--479.

[2]

D. Brezeale and D. J. Cook. 2008. Automatic Video Classification: A Survey of the Literature. Trans. Sys. Man Cyber Part C, Vol. 38, 3 (May 2008), 416--430. https://doi.org/10.1109/TSMCC.2008.919173

[3]

Paolo Cremonesi, Mehdi Elahi, and Franca Garzotto. 2017. User interface patterns in recommendation-empowered content intensive multimedia applications. Multimedia Tools and Applications, Vol. 76, 4 (01 Feb 2017), 5275--5309. https://doi.org/10.1007/s11042-016-3946-5

[4]

Peng Cui, Zhiyu Wang, and Zhou Su. 2014. What videos are similar with you? learning a common attributed representation for video recommendation. In Proceedings of the 22nd ACM international conference on Multimedia. 597--606.

Digital Library

[5]

Marco de Gemmis, Pasquale Lops, Cataldo Musto, Fedelucio Narducci, and Giovanni Semeraro. 2015. Semantics-Aware Content-Based Recommender Systems. In Recommender Systems Handbook. Springer, 119--159. https://doi.org/10.1007/978-1-4899-7637-6_4

[6]

Marco Degemmis, Pasquale Lops, and Giovanni Semeraro. 2007. A Content-collaborative Recommender That Exploits WordNet-based User Profiles for Neighborhood Formation. User Modeling and User-Adapted Interaction, Vol. 17, 3 (July 2007), 217--255 https://doi.org/10.1007/s11257-006-9023-4

Digital Library

[7]

Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, and Pietro Piazzolla. 2016a. Recommending Movies Based on Mise-En-Scene Design (CHI EA '16). Association for Computing Machinery, New York, NY, USA, 1540--1547. https://doi.org/10.1145/2851581.2892551

[8]

Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, Pietro Piazzolla, and Massimo Quadrana. 2016b. Content-Based Video Recommendation System Based on Stylistic Visual Features. Journal on Data Semantics (2016), 1--15. https://doi.org/10.1007/s13740-016-0060--9

[9]

Yashar Deldjoo, Mehdi Elahi, Massimo Quadrana, and Paolo Cremonesi. 2018. Using visual features based on MPEG-7 and deep learning for movie recommendation. International journal of multimedia information retrieval, Vol. 7, 4 (2018), 207--219. https://doi.org/10.1007/s13735-018-0155-1

[10]

Yashar Deldjoo, Mehdi Elahi, Massimo Quadrana, Paolo Cremonesi, and Franca Garzotto. 2015. Toward Effective Movie Recommendations Based on Mise-en-Scène Film Styles. In Proceedings of the 11th Biannual Conference on Italian SIGCHI Chapter. ACM, 162--165.

Digital Library

[11]

Farshad Bakhshandegan Moghaddam1 Mehdi Elahi et almbox. 2019. Cold Start Solutions For Recommendation Systems. IET.

[12]

Mehdi Elahi, Yashar Deldjoo, Farshad Bakhshandegan Moghaddam, Leonardo Cella, Stefano Cereda, and Paolo Cremonesi. 2017. Exploring the semantic gap for movie recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems. 326--330.

Digital Library

[13]

Mehdi Elahi, Francesco Ricci, and Neil Rubens. 2016. A survey of active learning in collaborative filtering recommender systems. Computer Science Review, Vol. 20 (2016), 29--50.

Digital Library

[14]

Manuel Enrich, Matthias Braunhofer, and Francesco Ricci. 2013. Cold-Start Management with Cross-Domain Collaborative Filtering and Tags. In Proceedings of the 13th International Conference on E-Commerce and Web Technologies. Springer, 101--112. https://doi.org/10.1007/978-3-642-39878-0_10

[15]

Ignacio Fernández-Tobías and Ivá n Cantador. 2014. Exploiting Social Tags in Matrix Factorization Models for Cross-domain Collaborative Filtering. In Proceedings of the 1st Workshop on New Trends in Content-based Recommender Systems, Foster City, California, USA. 34--41.

[16]

Claudiu S Firan, Wolfgang Nejdl, and Raluca Paiu. 2007. The benefit of using tag-based profiles. In 2007 Latin American Web Conference (LA-WEB 2007). IEEE, 32--41.

[17]

Mouzhi Ge, Mehdi Elahi, Ignacio Fernaández-Tob'ias, Francesco Ricci, and David Massimo. 2015. Using Tags and Latent Factors in a Food Recommender System (DH '15). ACM, New York, NY, USA, 105--112. https://doi.org/10.1145/2750511.2750528

[18]

F Maxwell Harper and Joseph A Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst., Vol. 5, 4, Article Article 19 (Dec. 2015), bibinfonumpages19 pages. https://doi.org/10.1145/2827872

Digital Library

[19]

Weiming Hu, Nianhua Xie, Li, Xianglin Zeng, and Stephen Maybank. 2011. A Survey on Visual Content-Based Video Indexing and Retrieval. Trans. Sys. Man Cyber Part C, Vol. 41, 6 (Nov. 2011), 797--819. https://doi.org/10.1109/TSMCC.2011.2109710

[20]

Yehuda Koren and Robert Bell. 2011. Advances in Collaborative Filtering. Springer US, Boston, MA, 145--186. https://doi.org/10.1007/978-0-387-85820-3_5

[21]

Huizhi Liang, Yue Xu, Yuefeng Li, and Richi Nayak. 2009. Tag Based Collaborative Filtering for Recommender Systems. In Rough Sets and Knowledge Technology, 4th International Conference, RSKT 2009, Gold Coast, Australia, July 14-16, 2009. Proceedings. 666--673. https://doi.org/10.1007/978-3-642-02962-2_84

[22]

Pasquale Lops, Marco de Gemmis, and Giovanni Semeraro. 2011. Content-based Recommender Systems: State of the Art and Trends. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul Kantor (Eds.). Springer, 73--105.

[23]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, Nov (2008), 2579--2605.

[24]

Marcelo Garcia Manzato. 2013. GSVD: Supporting Implicit Feedback on Recommender Systems with Metadata Awareness (SAC '13). Association for Computing Machinery, New York, NY, USA, 908--913. https://doi.org/10.1145/2480362.24805

Digital Library

[25]

Pablo Messina, Vicente Dominquez, Denis Parra, Christoph Trattner, and Alvaro Soto. 2018. Exploring Content-based Artwork Recommendation with Metadata and Visual Features. User Modeling and User-Adapted Interaction (UMUAI), Vol. 29, 2 (July 2018), 251--290. https://doi.org/10.1007/s11257-018-9206-9

[26]

Farshad B Moghaddam, Mehdi Elahi, Reza Hosseini, Christoph Trattner, and Marko Tkalvc ivc. 2019. Predicting Movie Popularity and Ratings with Visual Features. In 2019 14th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP). IEEE, 1--6. https://doi.org/10.1109/SMAP.2019.8864912

[27]

Reyn Nakamoto, Shinsuke Nakajima, Jun Miyazaki, and Shunsuke Uemura. 2007. Tag-based contextual collaborative filtering. In Proceedings of the 18th IEICE Data Engineering Workshop. 377--386.

[28]

H. R. Naphide and Thomas Huang. 2001. A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Transactions on Multimedia, Vol. 3, 1 (March 2001), 141--151. https://doi.org/10.1109/6046.909601

Digital Library

[29]

Zhou Qingbiao, Fang Jie, and Gaundong Xu. 2011. Incorporating sentiment analysis for improved tag-based recommendation. In 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing. IEEE, 1222--1227.

Digital Library

[30]

Zeeshan Rasheed, Yaser Sheikh, and Mubarak Shah. 2005. On the Use of Computable Features for Film Classification. IEEE Trans. Cir. and Sys. for Video Technol., Vol. 15, 1 (Jan. 2005), 52--64. https://doi.org/10.1109/TCSVT.2004.839993

Digital Library

[31]

Mohammad Hossein Rimaz, Mehdi Elahi, Farshad Bakhshandegan Moghadam, Christoph Trattner, Reza Hosseini, and Marko Tkalvcivc. 2019. Exploring the Power of Visual Features for the Recommendation of Movies (UMAP '19). Association for Computing Machinery, New York, NY, USA, 303--308. https://doi.org/10.1145/3320435.3320470

[32]

Neil Rubens, Mehdi Elahi, Masashi Sugiyama, and Dain Kaplan. 2015. Active Learning in Recommender Systems. In Recommender Systems Handbook. Springer, 809--846.

[33]

Jose San Pedro and Stefan Siersdorfer. 2009. Ranking and Classifying Attractiveness of Photos in Folksonomies (WWW '09). Association for Computing Machinery, New York, NY, USA, 771--780. https://doi.org/10.1145/1526709.1526813

[34]

Markus Schedl, Hamed Zamani, Ching-Wei Chen, Yashar Deldjoo, and Mehdi Elahi. 2018. Current challenges and visions in music recommender systems research. International Journal of Multimedia Information Retrieval, Vol. 7, 2 (2018), 95--116.

[35]

Cees G.M. Snoek and Marcel Worring. 2005. Multimodal Video Indexing: A Review of the State-of-the-art. Multimedia Tools and Applications, Vol. 25, 1 (01 Jan 2005), 5--35. https://doi.org/10.1023/B:MTAP.0000046380.27575.a5

[36]

Michele Svanera, Mattia Savardi, Alberto Signoroni, Andrá s Bá lint Ková cs, and Sergio Benini. 2018. Who is the director of this movie? Automatic style recognition based on shot features. CoRR, Vol. abs/1807.09560 (2018). arxiv: 1807.09560 http://arxiv.org/abs/1807.09560

[37]

Nava Tintarev and Judith Masthoff. 2011. Designing and evaluating explanations for recommender systems. In Recommender systems handbook. Springer, 479--510.

[38]

Marko Tkalvcivc, Nima Maleki, Matevvz Pesek, Mehdi Elahi, Francesco Ricci, and Matija Marolt. 2017. A Research Tool for User Preferences Elicitation with Facial Expressions (RecSys '17). ACM, New York, NY, USA, 353--354. https://doi.org/10.1145/3109859.3109978

[39]

Marko Tkalvcivc, Nima Maleki, Matevvz Pesek, Mehdi Elahi, Francesco Ricci, and Matija Marolt. 2019. Prediction of Music Pairwise Preferences from Facial Expressions (IUI '19). ACM, New York, NY, USA, 150--159. https://doi.org/10.1145/3301275.3302266

[40]

Christian Wartena, Rogier Brussee, and Martin Wibbels. 2009. Using tag co-occurrence for recommendation. In 2009 Ninth International Conference on Intelligent Systems Design and Applications. IEEE, 273--278.

Digital Library

[41]

Deqing Yang, Yanghua Xiao, Yangqiu Song, Junjun Zhang, Kezun Zhang, and Wei Wang. 2014. Tag propagation based recommendation across diverse social media. In Proceedings of the 23rd International Conference on World Wide Web. 407--408.

Digital Library

[42]

Zi-Ke Zhang, Chuang Liu, Yi-Cheng Zhang, and Tao Zhou. 2010. Solving the cold-start problem in recommender systems with social tags. EPL (Europhysics Letters), Vol. 92, 2 (2010), 28002.

Cited By

San Sequeira Dr. Amol Jogalekar (2023)Exploring the Use of Facial Attributes in Personality-Driven Recommendation Systems (FABaRS): A SurveyInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-8306(329-337)Online publication date: 9-Feb-2023
https://doi.org/10.48175/IJARSCT-8306
Lubos SFelfernig ATautschnig M(2023)An overview of video recommender systems: state-of-the-art and research issuesFrontiers in Big Data10.3389/fdata.2023.12816146Online publication date: 30-Oct-2023
https://doi.org/10.3389/fdata.2023.1281614
Adesh ADeepak GSanthanavijayan A(2023)SemVidRec: A Semantic Approach to Annotations Driven Video Recommendation Model Incorporating Machine IntelligenceComputational Intelligence and Network Systems10.1007/978-3-031-48984-6_1(1-13)Online publication date: 16-Dec-2023
https://doi.org/10.1007/978-3-031-48984-6_1
Show More Cited By

Index Terms

Visually-Aware Video Recommendation in the Cold Start
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Recommending Videos in Cold Start With Automatic Visual Tags
UMAP '21: Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization

This paper addresses the so-called New Item problem in video Recommender Systems, as part of Cold Start. New item problem occurs when a new item is added to the system catalog, and the recommender system has no or little data describing that item. This ...
Naïve filterbots for robust cold-start recommendations
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

The goal of a recommender system is to suggest items of interest to a user based on historical behavior of a community of users. Given detailed enough history, item-based collaborative filtering (CF) often performs as well or better than almost any ...
Tied boltzmann machines for cold start recommendations
RecSys '08: Proceedings of the 2008 ACM conference on Recommender systems

We describe a novel statistical model, the tied Boltzmann machine, for combining collaborative and content information for recommendations. In our model, pairwise interactions between items are captured through a Boltzmann machine, whose parameters are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HT '20: Proceedings of the 31st ACM Conference on Hypertext and Social Media

July 2020

327 pages

ISBN:9781450370981

DOI:10.1145/3372923

General Chairs:
Anastasia Salter
University of Central Florida, USA
,
John F. Barber
Washington State University Vancouver, USA
,
Program Chair:
Ujwal Gadiraju
Delft University of Technology, The Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HT '20

Sponsor:

HT '20: 31st ACM Conference on Hypertext and Social Media

July 13 - 15, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 378 of 1,158 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
254
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

San Sequeira Dr. Amol Jogalekar (2023)Exploring the Use of Facial Attributes in Personality-Driven Recommendation Systems (FABaRS): A SurveyInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-8306(329-337)Online publication date: 9-Feb-2023
https://doi.org/10.48175/IJARSCT-8306
Lubos SFelfernig ATautschnig M(2023)An overview of video recommender systems: state-of-the-art and research issuesFrontiers in Big Data10.3389/fdata.2023.12816146Online publication date: 30-Oct-2023
https://doi.org/10.3389/fdata.2023.1281614
Adesh ADeepak GSanthanavijayan A(2023)SemVidRec: A Semantic Approach to Annotations Driven Video Recommendation Model Incorporating Machine IntelligenceComputational Intelligence and Network Systems10.1007/978-3-031-48984-6_1(1-13)Online publication date: 16-Dec-2023
https://doi.org/10.1007/978-3-031-48984-6_1
de Jesus Oliveira VRottermanner GGrößbacher SBoucher MJudmaier P(2022)Requirements and Concepts for Interactive Media Retrieval User InterfacesNordic Human-Computer Interaction Conference10.1145/3546155.3546701(1-10)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546701
Huang CJiang T(2022)A Study of Danmaku Video Recommendation Algorithm Incorporating Multiple Features2022 4th International Conference on Frontiers Technology of Information and Computer (ICFTIC)10.1109/ICFTIC57696.2022.10075124(440-444)Online publication date: 2-Dec-2022
https://doi.org/10.1109/ICFTIC57696.2022.10075124
Abdullah NRasheed RNasir MRahman M(2021)Eliciting Auxiliary Information for Cold Start User Recommendation: A SurveyApplied Sciences10.3390/app1120960811:20(9608)Online publication date: 15-Oct-2021
https://doi.org/10.3390/app11209608
Karimpour DZare Chahooki MHashemi A(2021)Telegram group recommendation based on users' migration2021 26th International Computer Conference, Computer Society of Iran (CSICC)10.1109/CSICC52343.2021.9420581(1-6)Online publication date: 3-Mar-2021
https://doi.org/10.1109/CSICC52343.2021.9420581
Hazrati NElahi M(2020)Addressing the New Item problem in video recommender systems by incorporation of visual features with restricted Boltzmann machinesExpert Systems10.1111/exsy.1264538:3Online publication date: 19-Oct-2020
https://doi.org/10.1111/exsy.12645
Rimaz MHosseini RElahi MMoghaddam F(2020)AudioLens: Audio-Aware Video Recommendation for Mitigating New Item ProblemService-Oriented Computing – ICSOC 2020 Workshops10.1007/978-3-030-76352-7_35(365-378)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.1007/978-3-030-76352-7_35

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten