research-article

Content-Based Video Relevance Prediction with Second-Order Relevance and Attention Modeling

Authors:

Zheng-Jun ZhaAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 2018 - 2022

https://doi.org/10.1145/3240508.3266434

Published: 15 October 2018 Publication History

Get Access

Abstract

This paper describes our proposed method for the Content-Based Video Relevance Prediction (CBVRP) challenge. Our method is based on deep learning, i.e. we train a deep network to predict the relevance between two video sequences from their features. We explore the usage of second-order relevance, both in preparing training data, and in extending the deep network. Second-order relevance refers to e.g. the relevance between x and z if x is relevant to y and y is relevant to z. In our proposed method, we use second-order relevance to increase positive samples and decrease negative samples, when preparing training data. We further extend the deep network with an attention module, where the attention mechanism is designed for second-order relevant video sequences. We verify the effectiveness of our method on the validation set of the CBVRP challenge.

References

[1]

Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan. 2016. Youtube-8m: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675. (2016).

Google Scholar

[2]

Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, Pietro Piazzolla, and Massimo Quadrana. 2016. Content-based video recommendation system based on stylistic visual features. Journal on Data Semantics, Vol. 5, 2 (2016), 99--113.

Crossref

Google Scholar

[3]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In IEEE International Conference on Data Mining. 263--272.

Digital Library

Google Scholar

[4]

Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale video classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition. 1725--1732.

Digital Library

Google Scholar

[5]

Yan Li, Hanjie Wang, Hailong Liu, and Bo Chen. 2017. A study on content-based video recommendation. In IEEE International Conference on Image Processing. IEEE, 4581--4585.

Crossref

Google Scholar

[6]

Mengyi Liu, Xiaohui Xie, and Hanning Zhou. 2018. Content-based Video Relevance Prediction Challenge: Data, Protocol, and Baseline. arXiv preprint arXiv:1806.00737. (2018).

Google Scholar

[7]

Tao Mei, Bo Yang, Xian-Sheng Hua, and Shipeng Li. 2011. Contextual video recommendation by multimodal relevance and user feedback. ACM Transactions on Information Systems, Vol. 29, 2 (2011), 10.

Digital Library

Google Scholar

[8]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. ACM, 285--295.

Digital Library

Google Scholar

[9]

Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3D convolutional networks. In IEEE International Conference on Computer Vision. 4489--4497.

Digital Library

Google Scholar

[10]

Chong Wang and David M Blei. 2011. Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 448--456.

Digital Library

Google Scholar

[11]

Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1235--1244.

Digital Library

Google Scholar

[12]

Yin Zheng, Bangsheng Tang, Wenkui Ding, and Hanning Zhou. 2016. A neural autoregressive approach to collaborative filtering. In International Conference on Machine Learning .

Digital Library

Google Scholar

[13]

Qiusha Zhu, Mei-Ling Shyu, and Haohong Wang. 2013. Videotopic: Content-based video recommendation using a topic model. In IEEE International Symposium on Multimedia. IEEE, 219--222.

Digital Library

Google Scholar

Cited By

View all

Shang YGao CChen JJin DWang MLi YChen HDuh WHuang HKato MMothe JPoblete B(2023)Learning Fine-grained User Interests for Micro-video RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591713(433-442)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591713
Bhuyan HBrahma BRao P(2022)Multi-level feature learning approaches for video recommendation2022 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC)10.1109/ASSIC55218.2022.10088325(1-6)Online publication date: 19-Nov-2022
https://doi.org/10.1109/ASSIC55218.2022.10088325
Jiang HWang WWei YGao ZWang YNie LWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)What Aspect Do You LikeProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3413653(3487-3495)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3413653
Show More Cited By

Index Terms

Content-Based Video Relevance Prediction with Second-Order Relevance and Attention Modeling
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Interactive content-based image retrieval using relevance feedback

Database search engines are generally used in a one-shot fashion in which a user provides query information to the system and, in return, the system provides a number of database instances to the user. A relevance feedback system allows the user to ...
Multi-class relevance feedback content-based image retrieval

Relevance feedback methods for content-based image retrieval have shown promise in a variety of image database applications. These techniques assume two-class relevance feedback: relevant and irrelevant classes. While simple computationally, two-class ...
Content-Based Image Retrieval by Relevance Feedback
VISUAL '00: Proceedings of the 4th International Conference on Advances in Visual Information Systems

Relevance feedback is a powerful technique for content-based image retrieval. Many parameter estimation approaches have been proposed for relevance feedback. However, most of them have only utilized information of the relevant retrieved images, and have ...

Comments

Information & Contributors

Information

Published In

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
309
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Shang YGao CChen JJin DWang MLi YChen HDuh WHuang HKato MMothe JPoblete B(2023)Learning Fine-grained User Interests for Micro-video RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591713(433-442)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591713
Bhuyan HBrahma BRao P(2022)Multi-level feature learning approaches for video recommendation2022 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC)10.1109/ASSIC55218.2022.10088325(1-6)Online publication date: 19-Nov-2022
https://doi.org/10.1109/ASSIC55218.2022.10088325
Jiang HWang WWei YGao ZWang YNie LWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)What Aspect Do You LikeProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3413653(3487-3495)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3413653
Wang XDu YZhang LLi XZhang MDong JAmsaleg LHuet BLarson MGravier GHung HNgo CTsang Ooi W(2019)Exploring Content-based Video Relevance for Video Click-Through Rate PredictionProceedings of the 27th ACM International Conference on Multimedia10.1145/3343031.3356053(2602-2606)Online publication date: 15-Oct-2019
https://dl.acm.org/doi/10.1145/3343031.3356053
Dong JWang XZhang LXu CYang GLi X(2019)Feature Re-Learning with Data Augmentation for Video Relevance PredictionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2019.2947442(1-1)Online publication date: 2019
https://doi.org/10.1109/TKDE.2019.2947442

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Interactive content-based image retrieval using relevance feedback

Multi-class relevance feedback content-based image retrieval

Content-Based Image Retrieval by Relevance Feedback

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations