A framework for key-frame selection based on relevance feedback
Pages 259 - 264
Abstract
With the explosive growth of video resource, efficient techniques for generating video summarization are appealing for facilitating understanding and presenting video content. Traditional video summarizations were usually given through extracting key frames based on the features of frames in video sequence. However, in many cases, the given key frames don't meet the key frames reside in the mind of users. In this paper, we propose an innovative approach based on relevance feedback to select the key frames of a video sequence for video summarization considering users' subjective visual preference. A two-step strategy to select the key frames is given. 1) we evaluate the preference of user, and a Bayesian method is used to update the probability in condition of all the responses; 2) we take the interaction of the frames into consideration and select a proper frame set for video summarization. We verified that the relevance in different people's mind is not totally irrelevant. A relevance distance based on the characteristics of the video frames and the trend of users' decision making is proposed for more accurate likelihood definition. Experiments showed that our approach could provide satisfied summarizations in acceptable iteration in most cases and demonstrated the efficiency of the interactive and feedback process.
References
[1]
Barnes, C., Goldman D. B., Shechtman E., and Finkelstein A. 2010. Video Tapestries with Continuous Temporal Zoom. ACM Transactions on Graphics (TOG) 29, 4 Article 89.
[2]
Cheng, M.-M., Zhang, G.-X., Mitra N. J., Huang X., and Hu S.-M. 2011. Global Contrast Based Salient Region Detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011), 409--416
[3]
Cong, Y., Yuan, J., and Luo, J. 2012. Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection. IEEE Trans. On Multimedia, 14, 1, 66--75.
[4]
Correa, C. D., and Ma, K.-L. 2010. Dynamic Video Narratives. ACM Transactions on Graphics (TOG) 29, 4, Article 88.
[5]
Ferecatu, M., and Geman D. 2009. A Statistical Framework for Image Category Search From A Mental Picture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 6, 1087--1101.
[6]
Fu, Q.-F., Liu, Y-J., Chen, W. F., and Fu, X.-L. 2013 The Time Course of Natural Scene Categorization in Human Brain: Simple Line-drawings vs. Color Photographs. Journal of Vision 13, 9, 1060.
[7]
Girgensohn, A., and Boreczky J. 1999. Time-constrained Keyframe Selection Technique. In IEEE International Conference on Multimedia Computing and Systems.
[8]
Lienhart, R., Pfeiffer S., and Effelsberg W. 1997. Video Aabstracting. Communications of the ACM 40, 12, 54--62.
[9]
Liu, Y.-J., Luo., X., Joneja., A. Ma, C.-X. Fu, X.-L., and Song., D.-W. 2013. User-adaptive Sketch-based 3-D CAD Model Retrieval. IEEE Transactions on Automation Science and Engineering. 10, 3, 783--795.
[10]
Liu, Y.-J., Luo., X., Xuan, Y.-M., Chen, W.-F., and Fu, X.-L. 2011 Image Retargeting Quality Assessment. Computer Graphics Forum (Regular issue of Eurographics 2011), Vol. 30, No. 2, 583--592.
[11]
Liu, Y.-J., Fu, Q. -F., Liu, Y., and Fu, X.-L. 2013 A Distributed Computational Cognitive Model for Object Recognition. Science China (Series F: Information Sciences), Vol. 56, No. 9, Article No. 09210113), 1--13.
[12]
Ma, C.-X., Liu Y.-J., Wang H.-A., Teng D.-X., and Dai G.-Z. 2012. Sketch-based Annotation and Visualization in Video Authoring. IEEE Transactions on Multimedia, 14, 4, 1153--1165
[13]
Ma, Y.-F., Lu, L., Zhang H.-J., and Li M. 2002. A User Attention Model for Video Summarization. In Proceedings of the tenth ACM international conference on Multimedia, 533--542.
[14]
Nguyen, C., Niu Y., and Liu F. 2012. Video Summagator: An Interface for Video Summarization and Navigation. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, 647--650
[15]
Pritch, Y., Rav-Acha A., and Peleg S. 2008. Nonchronological Video Synopsis and Indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 11, 1971--1984.
[16]
Truong, B. T., and Venkatesh S. 2007. Video Abstraction: A Systematic Review and Classification. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 3, 1, 3.
[17]
Wang, Y.-S., Lin, H.-C., Sorkine, O., and Lee, T.-Y. 2010. Motion-based Video Retargeting with Optimized Crop-and-warp. ACM Transactions on Graphics (TOG) 29, 4, Areticle 90.
[18]
Yang, B., Mei T., Sun, L.-F., Yang, S.-Q., and Hua, X.-S. 2008. Free-shaped Video Collage. Advances in Multimedia Modeling, Springer, 175--185.
[19]
Yang, Y., Nie, F., Xu, D., Luo, J., Zhuang, Y. and Pan, Y. 2012. A Multimedia Retrieval Framework Based on Semi-supervised Ranking and Relevance Feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 4, 723--742.
[20]
Yeung, M. M., and Yeo, B.-L. 1997. Video Visualization for Compact Presentation and Fast Browsing of Pictorial Content. IEEE Transactions on Circuits and Systems for Video Technology, 7, 5, 771--785.
[21]
Zhang, J.-K., Ma, C.-X., Liu, Y.-J. Fu, Q. -F., and Fu, X.-L. 2013, Collaborative Interaction for Videos on Mobile Devices Based On Sketch Gestures. Journal of Computer Science and Technology, Vol. 28, No. 5, 810--817.
[22]
Zhou, X. S., and Huang, T. S. 2003. Relevance Feedback in Image Retrieval: A Comprehensive Review. Multimedia systems 8, 6, 536--544.
[23]
Zhuang, Y.-T., Yang, Y., and Wu, F. 2008. Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-media Retrieval. IEEE Transactions on Multimedia, 10, 2, 221--229.
[24]
Zhuang, Y., Rui, Y. Huang, T. S., and Mehrotra, S. 1998. Adaptive Key Frame Extraction Using Unsupervised Clustering. In IEEE. International Conference on Image Processing, 1, 866--870
Index Terms
- A framework for key-frame selection based on relevance feedback
Recommendations
A unified relevance model for opinion retrieval
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementRepresenting the information need is the greatest challenge for opinion retrieval. Typical queries for opinion retrieval are composed of either just content words, or content words with a small number of cue "opinion" words. Both are inadequate for ...
Interactive relevance feedback with graded relevance and sentence extraction: simulated user experiments
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementResearch on relevance feedback (RFB) in information retrieval (IR) has given mixed results. Success in RFB seems to depend on the searcher's willingness to provide feedback and ability to identify relevant documents or query keys. The paper is based on ...
A query-based multi-document sentiment summarizer
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge managementReview websites, such as Epinions.com, which offer users a platform to share their opinions on diverse products and services, provide a valuable source of opinion-rich information. Browsing through archived reviews to locate different opinions on a ...
Comments
Information & Contributors
Information
Published In
November 2013
325 pages
ISBN:9781450325905
DOI:10.1145/2534329
- General Chairs:
- Charlie C. L. Wang,
- Nadia Magnenat-Thalmann,
- Zhigeng Pan
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 November 2013
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
VRCAI 2013
Sponsor:
VRCAI 2013: 12th ACM International Conference on Virtual Reality Continuum and Its Applications in Industry
November 17 - 19, 2013
Hong Kong, Hong Kong
Acceptance Rates
VRCAI '13 Paper Acceptance Rate 35 of 75 submissions, 47%;
Overall Acceptance Rate 51 of 107 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 120Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in