Article

Scenario based dynamic video abstractions using graph matching

Authors:

Sae HwangAuthors Info & Claims

MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia

Pages 810 - 819

https://doi.org/10.1145/1101149.1101328

Published: 06 November 2005 Publication History

Abstract

In this paper, we present scenario based dynamic video abstractions using graph matching. Our approach has two main components: multi-level scenario generations and dynamic video abstractions. Multi-level scenarios are generated by a graph-based video segmentation and a hierarchy of the segments. Dynamic video abstractions are accomplished by accessing the generated hierarchy level by level. The first step in the proposed approach is to segment a video into shots using Region Adjacency Graph (RAG). A RAG expresses spatial relationships among segmented regions of a frame. To measure the similarity between two consecutive RAGs, we propose a new similarity measure, called Graph Similarity Measure (GSM). Next, we construct a tree structure called scene tree based on the correlation between the detected shots. The correlation is computed by the GSM since it considers the relations between the detected shots properly. Multi-level scenarios which provide various levels of video abstractions are generated using the constructed scene tree. We provide two types of abstraction using multi-level scenarios: multi-level highlights and multi-length summarizations. Multi-level highlights are made by entire shots in each scenario level. To summarize a video in various lengths, we select key frames by considering temporal relationships among RAGs computed by the GSM. We have developed a system, called Automatic Video Analysis System (AVAS), by integrating the proposed techniques to show their effectiveness. The experimental results show that the proposed techniques are promising.

References

[1]

J. Adcock, A. Girgensohn, M. Cooper, T. Liu, E. Rieffel, and L. Wilcox. FXPAL Experiments for TRECVID 2004. In Proceedings of TRECVID 2004 Workshop, March 2004.

[2]

E. Ardizzone and M. Cascia. Automatic video database indexing and retrieval. Multimedia Tools and Applications, 4:29--56, 1997.

Digital Library

[3]

H. Bunke, P. Foggia, C. Guidobaldi, C. Sansone, and M. Vento. A Comparison of Algorithms for Maximum Common Subgraph on Randomly Connected Graphs. In SSPR/SPR, pages 123--132, 2002.

Digital Library

[4]

H. Bunke and K. Shearer. A Graph Distance Metric based on the Maximal Common Subgraph. Pattern Recognition Letters 19, pages 255--259, 1998.

Digital Library

[5]

Z. Cernekova, C. Nikou, and I. Pitas. Entropy metrics used for video summarization. In Proceedings of the 18th spring conference on Computer graphics, pages 73--82, 2002.

Digital Library

[6]

Chang Yuan and Yu-Fei Ma and Hong-Jiang Zhang. A Graph-Theoretic Approach to Video Object Segmentation in 2D+t Space. Technical report, MSR, March 2003.

[7]

Chong-Wah Ngo and Yu-Fei Ma and Hong-Jiang Zhang. Video summarization and scene detection by graph modeling. IEEE Transactions on Circuits and Systems for Video Technology, 15(2):296--305, February 2005.

Digital Library

[8]

D. Comaniciu and P. Meer. Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. Pattern Anal. Mach. Intell, 24(5):603--619, 2002.

Digital Library

[9]

N. D. Doulamis, A. D. Doulamis, Y. S. Avrithis, and S. D. Kollias. Video Content Representation using Optimal Extraction of Frames and Scenes. In Proceedings of the IEEE International Conference on Image Processing, pages 875--879, 1998.

[10]

P. Foggia, C. Sansone, and M.Vento. An improved algorithm for matching large graphs. Proc. of 3rd IAPR-TC15 Workshop on GRPR, pages 149--159, 2001.

[11]

Hsuan-Wei Chen and Jin-Hau Kuo and Wei-Ta Chu and Ja-Ling Wu. Action movies segmentation and summarization based on tempo analysis. In Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, pages 251--258, 2004.

Digital Library

[12]

C. Kim and J. Hwang. An integrated scheme for object-based video abstraction. In Proc. of ACM Multimedia 2000, pages 303--311, LA, CA, Oct. 2000.

Digital Library

[13]

B. Li and I. Sezan. Event detection and summarization in american football broadcast video. In Proc. of SPIE Conference on Storage and Retrieval for Multimedia Databases 2002, pages 202--213, San Jose, CA, January 2002.

[14]

Li Zhao and Wei Qi and Stan Z. Li and Shi-Qiang Yang and HongJiang Zhang. Key-frame extraction and shot retrieval using nearest feature line (NFL). In Proceedings of the ACM Multimedia 2000 Workshops, pages 217--220, 2000.

Digital Library

[15]

R. Lienhart. Abstracting home video automatically. In Proc. ACM Multimedia 99 (Part 2), pages 37--40, Orlando, FL, October 1999.

Digital Library

[16]

R. Lienhart and S. Pfeiffer. Video abstracting. Communications of the ACM, 40(12):55--62, December 1997.

Digital Library

[17]

J. J. McGregor. Backtrack Search Algorithms and the Maximal Common Subgraph Problem. Software Practice and Experience, 12:23--34, 1982.

[18]

O. Miller, E. Navon, and A. Averbuch. Tracking of Moving Objects based on Graph Edges Similarity. In Proc. of the ICME '03, pages 73--76, 2003.

Digital Library

[19]

C. Ngo, T. Pong, and H. Zhang. On clustering and retrieval of video shots. In Proc. of ACM Multimedia 2001, pages 51--60, Ottawa, Canada, Oct. 2001.

Digital Library

[20]

J. Oh and K. A. Hua. An Efficient and Cost-effective Technique for Browsing and Indexing Large Video Databases. In Proc. of 2000 ACM SIGMOD Intl. Conf. on Management of Data, pages 415--426, Dallas, TX, May 2000.

Digital Library

[21]

N. Omoigui, L. He, A. Gupta, J. Grudin, and E. Sanocki. Time-Compression: Systems Concerns, Usage, and Benefits. In Proceeding of the CHI Conference on Human Factors in Computing Systems, pages 136--143, 1999.

Digital Library

[22]

Z. Rasheed and M. Shah. Scene Detection In Hollywood Movies and TV Shows. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 343--350, Madison, WI, 2003.

[23]

F. Shipman, A. Girgensohn, and L. Wilcox. Generation of interactive multi-level video summaries. In Proceedings of the eleventh ACM international conference on Multimedia, pages 392--401, 2003.

Digital Library

[24]

M. A. Smith and T. Kanade. Video Skimming and Characterization through the Combination of Image and Language Understanding. In Proc. of International Workshop on Content-Based Access of Image and Video Databases, pages 61--70, 1998.

Digital Library

[25]

C. Taskiran, A. Amir, and D. Ponceleon. Automated video summarization using speech transcripts. In Proc. of SPIE Conference on Storage and Retrieval for Multimedia Databases 2002, pages 371--382, San Jose, CA, January 2002.

[26]

H. Yu and W. Wolf. A visual search system for video and image databases. In Proc. IEEE Int'l Conf. on Multimedia Computing and Systems, pages 517--524, Ottawa, Canada, June 1997.

Digital Library

[27]

H. J. Zhang, C. Y. Low, S. W. Smoliar, and J. H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proc. of ACM Multimedia '95, pages 15--24, San Francisco, CA, 1995.

Digital Library

Cited By

Demir MBozma H(2015)Video Summarization via Segments Summary GraphsProceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2015.140(1071-1077)Online publication date: 7-Dec-2015
https://dl.acm.org/doi/10.1109/ICCVW.2015.140
Xu LLiu HYan XLiao SZhang X(2015)Optimization method for trajectory combination in surveillance video synopsis based on genetic algorithmJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-015-0278-76:5(623-633)Online publication date: 5-Apr-2015
https://doi.org/10.1007/s12652-015-0278-7
Zhu XLiu JWang JLu H(2014)Key observation selection-based effective video synopsis for camera networkMachine Vision and Applications10.1007/s00138-013-0519-825:1(145-157)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.1007/s00138-013-0519-8
Show More Cited By

Index Terms

Scenario based dynamic video abstractions using graph matching
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Video segmentation
      2. Computer vision tasks
        Scene understanding
        Video summarization

Recommendations

Region correspondence by inexact attributed planar graph matching
ICCV '95: Proceedings of the Fifth International Conference on Computer Vision

An efficient graph matching approach is proposed for finding region correspondence between two images of the same scene but taken from different viewpoints. Regions and their relations in an image are represented with region adjacency graph (RAG), which ...
Graph-based multilevel temporal video segmentation

This paper presents a graph-based multilevel temporal video segmentation method. In each level of the segmentation, a weighted undirected graph structure is implemented. The graph is partitioned into clusters which represent the segments of a video. ...
An Efficient Method for Computing Similarity Between Frequent Subgraphs
CGC '13: Proceedings of the 2013 International Conference on Cloud and Green Computing

Frequent sub graph mining and graph similarity measures are fundamental and prominent graph analytical techniques. These techniques are often applied together in many graph mining techniques such as clustering and classification. However, these ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia

November 2005

1110 pages

ISBN:1595930442

DOI:10.1145/1101149

General Chairs:
Hongjiang Zhang
Microsoft Research Asia, China
,
Tat-Seng Chua
National University of Singapore, Singapore
,
Program Chairs:
Ralf Steinmetz
Technische Universitat Darmstadt, Germany
,
Mohan Kankanhalli
National University of Singapore, Singapore
,
Lynn Wilcox
FXPAL

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM05

Sponsor:

MM05: 2005 13th Annual ACM International Conference on Multimedia

November 6 - 11, 2005

Hilton, Singapore

Acceptance Rates

MULTIMEDIA '05 Paper Acceptance Rate 49 of 312 submissions, 16%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
566
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Demir MBozma H(2015)Video Summarization via Segments Summary GraphsProceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2015.140(1071-1077)Online publication date: 7-Dec-2015
https://dl.acm.org/doi/10.1109/ICCVW.2015.140
Xu LLiu HYan XLiao SZhang X(2015)Optimization method for trajectory combination in surveillance video synopsis based on genetic algorithmJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-015-0278-76:5(623-633)Online publication date: 5-Apr-2015
https://doi.org/10.1007/s12652-015-0278-7
Zhu XLiu JWang JLu H(2014)Key observation selection-based effective video synopsis for camera networkMachine Vision and Applications10.1007/s00138-013-0519-825:1(145-157)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.1007/s00138-013-0519-8
Xu JJagadeesh VNi ZSunderrajan SManjunath B(2013)Graph-Based Topic-Focused Retrieval in Distributed Camera NetworkIEEE Transactions on Multimedia10.1109/TMM.2013.228101915:8(2046-2057)Online publication date: Dec-2013
https://doi.org/10.1109/TMM.2013.2281019
Chen QWang MHuang ZHua YSong ZYan S(2013)VideoPuzzleIEEE Transactions on Multimedia10.1109/TMM.2012.223630615:3(521-534)Online publication date: 1-Apr-2013
https://dl.acm.org/doi/10.1109/TMM.2012.2236306
Wang MHong RYuan XYan SChua T(2012)Movie2ComicsIEEE Transactions on Multimedia10.1109/TMM.2012.218718114:3(858-870)Online publication date: 1-Jun-2012
https://dl.acm.org/doi/10.1109/TMM.2012.2187181
Zhou XZhou XChen LBouguettaya A(2012)Efficient subsequence matching over large video databasesThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-011-0255-521:4(489-508)Online publication date: 1-Aug-2012
https://dl.acm.org/doi/10.1007/s00778-011-0255-5
Fu YGuo YZhu YLiu FSong CZhou Z(2010)Multi-View Video SummarizationIEEE Transactions on Multimedia10.1109/TMM.2010.205202512:7(717-729)Online publication date: 1-Nov-2010
https://dl.acm.org/doi/10.1109/TMM.2010.2052025
Zhou XZhou XChen LShu YBouguettaya ATaylor J(2010)Adaptive Subspace Symbolization for Content-Based Video DetectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2009.17122:10(1372-1387)Online publication date: 1-Oct-2010
https://dl.acm.org/doi/10.1109/TKDE.2009.171
Gao YWang WYong JGu H(2009)Dynamic video summarization using two-level redundancy detectionMultimedia Tools and Applications10.1007/s11042-008-0236-x42:2(233-250)Online publication date: 1-Apr-2009
https://dl.acm.org/doi/10.1007/s11042-008-0236-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten