skip to main content
10.1145/3240323.3240329acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

Video recommendation using crowdsourced time-sync comments

Published: 27 September 2018 Publication History

Abstract

Most existing work on video recommendation focuses on recommending a video as a whole, largely due to the unavailability of semantic information on video shot-level. Recently a new type of video comments has emerged, called time-sync comments, that are posted by users in real playtime of a video, thus each has a timestamp relative to the video playtime. In the present paper, we propose to utilize time-sync comments for three research tasks that are infeasible or difficult to tackle in the past, namely (1) video clustering based on temporal user emotional/topic trajectory inside a video; (2) video highlight shots recommendation unsupervisedly; (3) personalized video shot recommendation tailored to user moods. We analyze characteristics of time-sync comments, and propose feasible solutions for each research task. For task (1), we propose a deep recurrent auto-encoder framework coupled with dictionary learning to model user emotional/topical trajectories in a video. For task (2), we propose a scoring method based on emotional/topic concentration in time-sync comments for candidate highlight shot ranking. For task (3), we propose a joint deep collaborative filtering network that optimizes ranking loss and classification loss simultaneously. Evaluation methods and preliminary experimental results are also reported. We plan to further refine our models for task (1) and (3) as our next step.

References

[1]
Kevin Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. 1999. When is "nearest neighbor" meaningful?. In International conference on database theory. Springer, 217--235.
[2]
Snigdha Chaturvedi, Mohit Iyyer, and Hal Daumé III. 2017. Unsupervised Learning of Evolving Relationships Between Literary Characters. In AAAI. 3159--3165.
[3]
Xu Chen, Yongfeng Zhang, Qingyao Ai, Hongteng Xu, Junchi Yan, and Zheng Qin. 2017. Personalized key frame recommendation. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 315--324.
[4]
James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, et al. 2010. The YouTube video recommendation system. In Proceedings of the fourth ACM conference on Recommender systems. ACM, 293--296.
[5]
Lea Frermann and György Szarvas. 2017. Inducing semantic micro-clusters from deep multi-view representations of novels. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 1873--1883.
[6]
Alan Hanjalic and Li-Qun Xu. 2005. Affective video content representation and modeling. IEEE transactions on multimedia 7, 1 (2005), 143--154.
[7]
Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal Daumé III. 2016. Feuding families and former friends: Unsupervised learning for dynamic fictional relationships. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1534--1544.
[8]
Keng-Sheng Lin, Ann Lee, Yi-Hsuan Yang, Cheng-Te Lee, and Homer H Chen. 2013. Automatic highlights extraction for drama video using music emotion and human face features. Neurocomputing 119 (2013), 111--117.
[9]
Guangyi Lv, Tong Xu, Enhong Chen, Qi Liu, and Yi Zheng. 2016. Reading the Videos: Temporal Labeling for Crowdsourced Time-Sync Videos Based on Semantic Embedding. In AAAI. 3000--3006.
[10]
Chong-Wah Ngo, Yu-Fei Ma, and Hong-Jiang Zhang. 2005. Video summarization and scene detection by graph modeling. IEEE Transactions on Circuits and Systems for Video Technology 15, 2 (2005), 296--305.
[11]
Juan Carlos Niebles, Hongcheng Wang, and Li Fei-Fei. 2008. Unsupervised learning of human action categories using spatial-temporal words. International journal of computer vision 79, 3 (2008), 299--318.
[12]
Qing Ping and Chaomei Chen. 2017. Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowdsourced Time-Sync Comments. In Proceedings of the Workshop on New Frontiers in Summarization. 1--11.
[13]
Arash Vahdat, Guang-Tong Zhou, and Greg Mori. 2014. Discovering video clusters from visual features and noisy tags. In European Conference on Computer Vision. Springer, 526--539.
[14]
Jingya Wang, Xiatian Zhu, and Shaogang Gong. 2016. Video Semantic Clustering with Sparse and Incomplete Tags. In AAAI. 3618--3624.
[15]
Yikun Xian, Jiangfeng Li, Chenxi Zhang, and Zhenyu Liao. 2015. Video highlight shot extraction with time-sync comment. In Proceedings of the 7th International Workshop on Hot Topics in Planet-scale mObile computing and online Social neTworking. ACM, 31--36.
[16]
Min Xu, Jesse S Jin, Suhuai Luo, and Lingyu Duan. 2008. Hierarchical movie affective content analysis based on arousal and valence features. In Proceedings of the 16th ACM international conference on Multimedia. ACM, 677--680.
[17]
Min Xu, Suhuai Luo, Jesse S Jin, and Mira Park. 2009. Affective content analysis by mid-level representation in multiple modalities. In Proceedings of the First International Conference on Internet Multimedia Computing and Service. ACM, 201--207.
[18]
Ming Yan, Jitao Sang, and Changsheng Xu. 2015. Unified youtube video recommendation via cross-network collaboration. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. ACM, 19--26.
[19]
Dong-Qing Zhang, Ching-Yung Lin, Shi-Fu Chang, and John R Smith. 2004. Semantic video clustering across sources using bipartite spectral clustering. In Multimedia and Expo, 2004. ICME'04. 2004 IEEE International Conference on, Vol. 1. IEEE, 117--120.
[20]
Guang-Tong Zhou, Tian Lan, Arash Vahdat, and Greg Mori. 2013. Latent maximum margin clustering. In Advances in Neural Information Processing Systems. 28--36.

Cited By

View all

Index Terms

  1. Video recommendation using crowdsourced time-sync comments

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems
    September 2018
    600 pages
    ISBN:9781450359016
    DOI:10.1145/3240323
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 September 2018

    Check for updates

    Author Tags

    1. mood-aware recommendation
    2. personalized video shot recommendation
    3. time-sync comments
    4. video clustering
    5. video highlight detection

    Qualifiers

    • Extended-abstract

    Conference

    RecSys '18
    Sponsor:
    RecSys '18: Twelfth ACM Conference on Recommender Systems
    October 2, 2018
    British Columbia, Vancouver, Canada

    Acceptance Rates

    RecSys '18 Paper Acceptance Rate 32 of 181 submissions, 18%;
    Overall Acceptance Rate 254 of 1,295 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)12
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Personalized time-sync comment generation based on a multimodal transformerMultimedia Systems10.1007/s00530-024-01301-330:2Online publication date: 30-Mar-2024
    • (2023)Sentiment Analysis on Online Videos by Time-Sync CommentsEntropy10.3390/e2507101625:7(1016)Online publication date: 2-Jul-2023
    • (2022)Qrator: An Interest-Aware Approach to ABR Streaming Based on User EngagementIEEE Systems Journal10.1109/JSYST.2022.315971916:4(6581-6589)Online publication date: Dec-2022
    • (2022)An Autonomous Data Collection Pipeline for Online Time-Sync Comments2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00053(327-336)Online publication date: Jun-2022
    • (2022)VisDmk: visual analysis of massive emotional danmaku in online videosThe Visual Computer10.1007/s00371-022-02748-z39:12(6553-6570)Online publication date: 24-Dec-2022
    • (2020)Live Stream Highlight Detection Using Chat Messages2020 21st IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM48529.2020.00072(328-332)Online publication date: Jun-2020
    • (2020)Learning Alone yet Together: An Exploratory Study of the Interaction through Online Danmaku Videos2020 IEEE Learning With MOOCS (LWMOOCS)10.1109/LWMOOCS50143.2020.9234326(100-104)Online publication date: 29-Sep-2020
    • (2020)TSCREC: Time-sync Comment Recommendation in Danmu-Enabled Videos2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI50040.2020.00021(67-72)Online publication date: Nov-2020
    • (2019)Interactive Variance Attention based Online Spoiler Detection for Time-Sync CommentsProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357872(1241-1250)Online publication date: 3-Nov-2019
    • (2019)Herding Effect Based Attention for Personalized Time-Sync Video Recommendation2019 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME.2019.00085(454-459)Online publication date: Jul-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media