skip to main content
10.1145/2660505.2660513acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Sentiment Flow for Video Interestingness Prediction

Published:07 November 2014Publication History

ABSTRACT

Computational analysis and prediction of digital media interestingness is a challenging task, largely driven by subjective nature of interestingness. Several attempts were made to construct a reliable measure and obtain a better understanding of interestingness based on various psychological study results. However, most current works focus on interestingness prediction for images. While the video affective analysis has been studied for quite some time, there are few works that explictly try to predict interestingness of videos. In this work, we extend a recent pilot study on the video interestingness prediction by using a mid-level representation of sentiment (emotion) sequence. We evaluate our proposed framework on three datasets including the datasets proposed by the pilot study and show that the result effectively verifies a promising utility of the approach.

References

  1. E. Acar. Learning Representations for A_ective Video Understanding. In ACM MM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. D. Berlyne. Conict, arousal, and curiosity. McGraw-Hill, 1960.Google ScholarGoogle Scholar
  3. I. Biederman and E. Vessel. Perceptual Pleasure and the Brain. American Scientist, 2006.Google ScholarGoogle ScholarCross RefCross Ref
  4. D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang. Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs. In ACM MM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. N. Dalal and B. Triggs. Histogram of oriented gradients for human detection. In IEEE CVPR, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. R. Datta, D. Joshi, J. Li, and J. Wang. Studying Aesthetics in Photographic Images Using a Computational Approach. In ECCV, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. Dhar, V. Ordonez, and T. Berg. High Level Describable Attributes for Predicting Aesthetics and Interestingness. In IEEE CVPR, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Gönen and E. Alpaydin. Multiple Kernel Learning Algorithms. JMLR, 12:2211--2268, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. H. Grabner, F. Nater, M. Druey, and L. V. Gool. Visual interestingness in image sequences. In ACM MM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Gygli, H. Grabner, H. Riemenschneider, F. Nater, and L. V. Gool. The interestingness of images. In ICCV, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. R. Herbrich, T. Graepel, and K. Obermayer. Large Margin Rank Boundaries for Ordinal Regression, chapter 7, pages 115--132. MIT Press, 2000.Google ScholarGoogle Scholar
  12. T. Jaakkola and D. Haussler. Exploiting generative models in discriminative classifiers. In NIPS, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Y.-G. Jiang, Y. Wang, R. Feng, X. Xue, Y. Zheng, and H. Yang. Understanding and predicting interestingness of videos. In AAAI, 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. S. Koelstra, C. Mühl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras. DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE Trans. Affective Computing, 3:18--31, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. L.-J. Li, H. Su, E. P. Xing, and L. Fei-Fei. Object Bank: A High-Level Image Representation for Scene Classification and Semantic Feature Sparsification. In NIPS, 2010.Google ScholarGoogle Scholar
  16. F. Liu, Y. Niu, and M. Gleicher. Using Web Photos for Measuring Video Frame Interestingness. In IJCAI, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91--110, Nov. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. Machajdik and A. Hanbury. A_ective image classification using features inspired by psychology and art theory. In ACM MM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. N. Murray, L. Marchesotti, and F. Perronnin. AVA: A Large-Scale Database for Aesthetic Visual Analysis. In IEEE CVPR, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  20. F. Perronnin, J. S_anchez, and T. Mensink. Improving the fisher kernel for large-scale image classification. In ECCV, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. Plutchik and H. Kellerman. Emotion: Theory, Research, and Experience Volume 1. Academic Press,Google ScholarGoogle Scholar
  22. T. Schaul, L. Pape, T. Glasmachers, V. Graziano, and J. Schmidhuber. Coherence Progress: A Measure of Interestingness Based on Fixed Compressors. In Artificial General Intelligence, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. K. R. Scherer. What are emotions? And how can they be measured? Social Science Information, 44:695--729, 2005.Google ScholarGoogle ScholarCross RefCross Ref
  24. E. Shechtman and M. Irani. Matching Local Self-Similarities across Images and Videos. In IEEE CVPR, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  25. M. Soleymani, G. Chanel, J. J. Kierkels, and T. Pun. Affective ranking of movie scenes using physiological signals and content analysis. In Proc. of the 2nd ACM Workshop on MM Semantics, MS '08, pages 32--39, New York, NY, USA, 2008. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. R. Srivastava, S. Yan, T. Sim, and S. Roy. Recognizing emotions of characters in movies. In IEEE ICASSP, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  27. L. Torresani, M. Szummer, and A. Fitzgibbon. Efficient Object Category Recognition Using Classemes. In ECCV, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Wang, C. Xu, E. Chng, and Q. Tian. Sports highlight detection from keyword sequences using hmm. In IEEE ICME, volume 1, pages 599--602 Vol.1, June 2004.Google ScholarGoogle Scholar
  29. M. Xu, J. Jin, S. Luo, and L. Duan. Hierarchical movie a_ective content analysis based on arousal and valence features. In ACM MM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Q. Zhang and B. Li. Relative hidden markov models for evaluating motion skills. In IEEE CVPR, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Sentiment Flow for Video Interestingness Prediction

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          HuEvent '14: Proceedings of the 1st ACM International Workshop on Human Centered Event Understanding from Multimedia
          November 2014
          58 pages
          ISBN:9781450331203
          DOI:10.1145/2660505

          Copyright © 2014 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 7 November 2014

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          HuEvent '14 Paper Acceptance Rate9of11submissions,82%Overall Acceptance Rate9of11submissions,82%

          Upcoming Conference

          MM '24
          MM '24: The 32nd ACM International Conference on Multimedia
          October 28 - November 1, 2024
          Melbourne , VIC , Australia

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader