skip to main content
10.1145/3206025.3206078acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
research-article

Deep Pairwise Classification and Ranking for Predicting Media Interestingness

Published: 05 June 2018 Publication History

Abstract

With the explosive increase in the consumption of multimedia content in recent years, the field of media interestingness analysis has gained a lot of attention. This paper tackles the problem of image interestingness in videos and proposes a novel algorithm based on pairwise-comparisons of frames to rank all frames in a video. Experiments performed on the Predicting Media Interestingness dataset, affirm its effectiveness over existing solutions. In terms of the official metric i.e. Mean Average Precision at 10, it outperforms the previous state-of-the-art (to the best of our knowledge) on this dataset. Additional results on video interestingness substantiate the flexibility and performance reliability of our approach.

References

[1]
Jurandy Almeida and Ricardo M Savii. 2017. GIBIS at MediaEval 2017: Predicting Media Interestingness Task Proc. of the MediaEval 2017 Workshop, Dublin, Ireland.
[2]
Yusuf Aytar, Carl Vondrick, and Antonio Torralba. 2016. Soundnet: Learning sound representations from unlabeled video Advances in Neural Information Processing Systems. 892--900.
[3]
Olfa Ben-Ahmed, Jonas Wacker, Alessandro Gaballo, and Benoit Huet. 2017. EURECOM@ MediaEval 2017: Media Genre Inference for Predicting Media Interestingnes Proc. of the MediaEval Workshop, Dublin, Ireland.
[4]
Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika Vol. 39, 3/4 (1952), 324--345.
[5]
Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard S"ackinger, and Roopak Shah. 1994. Signature verification using a siamese time delay neural network Advances in Neural Information Processing Systems. 737--744.
[6]
Franccois Chollet et almbox. 2015. Keras. (2015).
[7]
William W Cohen, Robert E Schapire, and Yoram Singer. 1998. Learning to order things. In Advances in Neural Information Processing Systems. 451--457.
[8]
Mihai Gabriel Constantin, Bogdan Boteanu, and Bogdan Ionescu. 2017. LAPI at MediaEval 2017-Predicting Media Interestingness Proc. of the MediaEval Workshop, Dublin, Ireland.
[9]
Claire-Helène Demarty, Mats Sjöberg, Mihai Gabriel Constantin, Ngoc QK Duong, Bogdan Ionescu, Thanh-Toan Do, and Hanli Wang. 2017 a. Predicting Interestingness of Visual Content. In Visual Content Indexing and Retrieval with Psycho-Visual Models. Springer, 233--265.
[10]
C.-H. Demarty, Mats Sjöberg, Bogdan Ionescu, Thanh-Toan Do, Michael Gygli, and Ngoc Q.K. Duong. 2017 b. MediaEval 2017 Predicting Media Interestingness Task Proc. of the MediaEval Workshop, Dublin, Ireland.
[11]
Claire-Hélène Demarty, Mats Viktor Sjöberg, Bogdan Ionescu, Thanh-Toan Do, Hanli Wang, Ngoc QK Duong, Frédéric Lefebvre, et almbox. 2016. Mediaeval 2016 predicting media interestingness task Proc. of the MediaEval Workshop, Hilversum, Netherlands.
[12]
Sagnik Dhar, Vicente Ordonez, and Tamara L Berg. 2011. High level describable attributes for predicting aesthetics and interestingness IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1657--1664.
[13]
Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, and César A Hidalgo. 2016. Deep learning the city: Quantifying urban perception at a global scale European Conference on Computer Vision. Springer, 196--212.
[14]
Michael Gygli, Helmut Grabner, Hayko Riemenschneider, Fabian Nater, and Luc Van Gool. 2013. The interestingness of images. In IEEE International Conference on Computer Vision (ICCV). IEEE, 1633--1640.
[15]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks Advances in Neural Information Processing Systems. 1097--1105.
[16]
Tie-Yan Liu et almbox. 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval Vol. 3, 3 (2009), 225--331.
[17]
Yang Liu, Zhonglei Gu, and Tobey H Ko. 2017. Predicting Media Interestingness via Biased Discriminant Embedding and Supervised Manifold Regression. In Proc. of the MediaEval Workshop, Dublin, Ireland.
[18]
Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines Proc. of the 27th International Conference on Machine Learning (ICML). 807--814.
[19]
Jayneel Parekh, Harshvardhan Tibrewal, and Sanjeel Parekh. 2017. The IITB Predicting Media Interestingness System for MediaEval 2017 Proc. of the MediaEval Workshop, Dublin, Ireland.
[20]
Devi Parikh and Kristen Grauman. 2011. Relative attributes. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 503--510.
[21]
Reza Aditya Permadi, Septian Gilang Permana Putra, Cynthia Helmiriawan, and CS Liem. 2017. DUT-MMSR at MediaEval 2017: Predicting Media Interestingness Task Proc. of the MediaEval Workshop, Dublin, Ireland.
[22]
K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR Vol. abs/1409.1556 (2014).
[23]
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the importance of initialization and momentum in deep learning International Conference on Machine Learning (ICML). 1139--1147.
[24]
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks IEEE International Conference on Computer Vision (ICCV). IEEE, 4489--4497.
[25]
Edward A Vessel and Nava Rubin. 2010. Beauty and the beholder: highly individual taste for abstract, but not real-world images. Journal of Vision Vol. 10, 2 (2010), 18--18.
[26]
Shuai Wang, Shizhe Chen, Jinming Zhao, Wenxuan Wang, and Qin Jin. 2017. RUC at MediaEval 2017: Predicting Media Interestingness Task Proc. of the MediaEval Workshop, Dublin, Ireland.
[27]
Guoqiang Peter Zhang. 2000. Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) Vol. 30, 4 (2000), 451--462.

Cited By

View all
  • (2024)Anomaly and Interestingness Detection in Timed Hierarchical Business ProcessesIEEE Transactions on Engineering Management10.1109/TEM.2022.318241371(12619-12634)Online publication date: 2024
  • (2021)Visual Interestingness Prediction: A Benchmark Framework and Literature ReviewInternational Journal of Computer Vision10.1007/s11263-021-01443-1129:5(1526-1550)Online publication date: 22-Feb-2021
  • (2021)Exploring Deep Fusion Ensembling for Automatic Visual Interestingness PredictionHuman Perception of Visual Information10.1007/978-3-030-81465-6_2(33-58)Online publication date: 22-Jul-2021
  • Show More Cited By

Index Terms

  1. Deep Pairwise Classification and Ranking for Predicting Media Interestingness

                                    Recommendations

                                    Comments

                                    Information & Contributors

                                    Information

                                    Published In

                                    cover image ACM Conferences
                                    ICMR '18: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval
                                    June 2018
                                    550 pages
                                    ISBN:9781450350464
                                    DOI:10.1145/3206025
                                    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

                                    Sponsors

                                    Publisher

                                    Association for Computing Machinery

                                    New York, NY, United States

                                    Publication History

                                    Published: 05 June 2018

                                    Permissions

                                    Request permissions for this article.

                                    Check for updates

                                    Author Tags

                                    1. artificial neural network
                                    2. image interestingness
                                    3. media interestingness
                                    4. pairwise comparisons

                                    Qualifiers

                                    • Research-article

                                    Conference

                                    ICMR '18
                                    Sponsor:

                                    Acceptance Rates

                                    ICMR '18 Paper Acceptance Rate 44 of 136 submissions, 32%;
                                    Overall Acceptance Rate 254 of 830 submissions, 31%

                                    Contributors

                                    Other Metrics

                                    Bibliometrics & Citations

                                    Bibliometrics

                                    Article Metrics

                                    • Downloads (Last 12 months)9
                                    • Downloads (Last 6 weeks)3
                                    Reflects downloads up to 30 Jan 2025

                                    Other Metrics

                                    Citations

                                    Cited By

                                    View all
                                    • (2024)Anomaly and Interestingness Detection in Timed Hierarchical Business ProcessesIEEE Transactions on Engineering Management10.1109/TEM.2022.318241371(12619-12634)Online publication date: 2024
                                    • (2021)Visual Interestingness Prediction: A Benchmark Framework and Literature ReviewInternational Journal of Computer Vision10.1007/s11263-021-01443-1129:5(1526-1550)Online publication date: 22-Feb-2021
                                    • (2021)Exploring Deep Fusion Ensembling for Automatic Visual Interestingness PredictionHuman Perception of Visual Information10.1007/978-3-030-81465-6_2(33-58)Online publication date: 22-Jul-2021
                                    • (2020)System Fusion with Deep EnsemblesProceedings of the 2020 International Conference on Multimedia Retrieval10.1145/3372278.3390720(256-260)Online publication date: 8-Jun-2020
                                    • (2019)Computational Understanding of Visual Interestingness Beyond SemanticsACM Computing Surveys10.1145/330129952:2(1-37)Online publication date: 27-Mar-2019

                                    View Options

                                    Login options

                                    View options

                                    PDF

                                    View or Download as a PDF file.

                                    PDF

                                    eReader

                                    View online with eReader.

                                    eReader

                                    Figures

                                    Tables

                                    Media

                                    Share

                                    Share

                                    Share this Publication link

                                    Share on social media