Yue Gao
ABSTRACT
In this paper, we present a new algorithm for video clip summarization and ranking, which is mainly based on a clip based video similarity measure and the affinity propagation clustering (AP) algorithm. We propose a proportional max-weighted bipartite matching algorithm for clip similarity measure. This method first generates a basic frame set and a corresponding proportion value set from each clip. Then it models two clips as a weighted bipartite graph, where the weight values are determined by both the direct frame similarities and the proportion values. Then the max-weighted bipartite matching is employed to measure the similarity between two clips. This method achieves good retrieval performance when the length of two clips varies greatly. With these clip similarities, clips are clustered using affinity propagation. The clips in one cluster generally describe the same video event. Video ranking is based on the cluster size and the average information entropy of each event. Experimental results are given to illustrate the proposed algorithm.
- J. Calic, D. Gibson, and N. Campbell. Efficient layout of comic-like video summaries. IEEE Transactions on Circuits and Systems for Video Technology, 17(7):931 -- 936, 2007. Google Scholar
Digital Library
- B. Frey and D. Dueck. Clustering by passing messages between data points. Science, 315:972 -- 976, 2007.Google Scholar
- J. Gauch and A. Shivadas. Identification of new commercials using repeated video sequence detection. In Proceeding of IEEE International Conference on Image Processing, pages 1252 -- 1255, 2005.Google ScholarCross Ref
- S. Guimaraes and R. Kelly. Counting of video clip repetitions using a modified bmh algorithm: Preliminary results. In Proceeding of IEEE International Conference on Multimedia Expo, pages 1065 -- 1068, 2006.Google Scholar
- A. Hanjalic. Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Transactions on Multimedia, 7(6):1114--1122, 2005. Google ScholarDigital Library
- Y. Ho, C. Lin, J. Chen, and H. Liao. Fast coarse-to-fine video retrieval using shot-level spatio-temporal statistics. IEEE Transactions on Circuits and Systems for Video Technology, 16(5):642 -- 648, 2006. Google ScholarDigital Library
- A. Jain, A. Vailaya, and X. Wei. Query by video clip. Multimedia Systems, 7(5):369 -- 384, 1999. Google ScholarDigital Library
- A. Joly, C. Frelicot, and O. Buisson. Content-based video copy detection in large data base. In Proceeding of IEEE International Conference on Image Processing, pages 505 -- 508, 2005.Google Scholar
- S. Kim and R. Park. An efficient algorithm for video sequence matching using the modified hausdorff distance and the directed divergence. IEEE Transactions on Circuits and Systems for Video Technology, 12(7):592 -- 596, 2002. Google ScholarDigital Library
- Y. Kim and T. Chua. Retrieval of news video using video sequence matching. In Proceeding of IEEE International Conference on Multimedia Modeling, pages 68 -- 75, 2005. Google ScholarDigital Library
- H. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2:83 -- 97, 1955.Google Scholar
- V. Kulesh, V. Petrushin, and I. Sethi. Video clip recognition using joint audio-visual processing model. In Proceeding of International Conference on Pattern Recognition, volume 1, pages 500 -- 503, 2002. Google ScholarDigital Library
- J. Lee, J. Oh, and S. Hwang. Scenario based dynamic video abstractions using graph matching. In Proceeding of ACM International Conference on Multimedia, pages 810 -- 819, 2005. Google ScholarDigital Library
- Z. Li, G. Schuster, and A. Katsaggelos. Rate-distortion optimal video summary generation. IEEE Transactions on Image Processing, 14(10):1550 -- 1560, 2005. Google ScholarDigital Library
- T. Liu and R. Katpelly. Content-adaptive video summarization combining queueing and clustering. In Proceeding of IEEE International Conference on Image Processing, pages 145 -- 148, 2006.Google ScholarCross Ref
- C. Ngo, Y. Ma, and H. Zhang. Video summarization and scene detection by graph modeling. IEEE Transactions on Circuits and Systems for Video Technology, 15(2):296 -- 305, 2005. Google ScholarDigital Library
- Y. Peng and C. Ngo. Hot event detection and summarization by graph modeling and matching. Lecture Notes in Computer Science, 3568:257 -- 266, 2005. Google ScholarDigital Library
- Y. Peng and C. Ngo. Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Transactions on Circuits and Systems for Video Technology, 16(5):612 -- 627, 2006. Google ScholarDigital Library
- F. Rothganger, S. Lazebnik, C. Schmid, and J. Ponce. Segmenting, modeling, and matching video clips containing multiple moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3):477 -- 491, 2007. Google ScholarDigital Library
- P. Sand and S. Teller. Video matching. ACM Transactions on Graphics, 23(3):592 -- 599, 2004. Google ScholarDigital Library
- C. Xu, X. Shao, N. Maddage, and M. Kankanhalli. Automatic music video summarization based on audio-visual-text analysis and alignment. In Proceeding of International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 361 -- 368, 2005. Google ScholarDigital Library
- J. Yuan, Q. Tian, and S. Ranganath. Fast and robust search method for short video clips from large video collection. In Proceeding of IEEE International Conference on Pattern Recognition, volume 3, pages 866 -- 869, 2004. Google ScholarDigital Library
Index Terms
- Clip based video summarization and ranking
Recommendations
Clip-based similarity measure for query-dependent clip retrieval and video summarization
This paper proposes a new approach and algorithm for the similarity measure of video clips. The similarity is mainly based on two bipartite graph matching algorithms: maximum matching (MM) and optimal matching (OM). MM is able to rapidly filter ...
Automatic music video summarization based on audio-visual-text analysis and alignment
SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrievalIn this paper, we propose a novel approach for automatic music video summarization based on audio-visual-text analysis and alignment. The music video is separated into the music and video tracks. For the music track, the chorus is detected based on ...
Applying two-level reinforcement ranking in query-oriented multidocument summarization
Sentence ranking is the issue of most concern in document summarization today. While traditional feature-based approaches evaluate sentence significance and rank the sentences relying on the features that are particularly designed to characterize the ...
Comments