skip to main content
10.1145/2461466.2461506acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
poster

A general Framework of video segmentation to logical unit based on conditional random fields

Published: 16 April 2013 Publication History

Abstract

Segmenting video into logical units like scenes in movies and topic units in News videos is an essential prerequisite for a wide range of video related applications. In this paper, a novel approach for logical unit segmentation based on conditional random fields (CRFs) is presented. In comparison with previous approaches that handle scenes and topic units separately, the proposed approach deals with them in a general framework. Specifically, four types of shots are defined and represented by four middle-level features, i.e., shot difference, scene transition, shot theme and audio type. Then, the problem of logical unit segmentation is novelly formulated as a problem of identifying the type of shot based on the extracted features, by leveraging the CRFs model. The proposed framework effectively integrate visual, audio and contextual features, and it is able to produce ideal result for both scene and topic unit segmentation. The effectiveness of the proposed approach is verified on seven mainstream types of videos, from which average F-measures of 88% and 86% on scenes and topic units are reported respectively, illustrating that the proposed method can accurately segment logical units in different genres of videos.

References

[1]
W. Ce, W. Yun, L. Hua-Yong, and H. Yan-Xiang. Automatic story segmentation of news video based on audio-visual features and text information. InMachine Learning and Cybernetics, 2003 International Conference on, volume 5, pages 3008{3011 Vol.5, 2003.
[2]
L. Chaisorn, T.-S. Chua, and C.-H. Lee. A multi-modal approach to story segmentation for news video.World Wide Web, 6(2):187--208, 2003.
[3]
V. T. Chasanis, A. C. Likas, and N. P. Galatsanos. Scene detection in videos using shot clustering and sequence alignment.Multimedia, IEEE Transactions on, 11(1):89--100, 2009.
[4]
N. Chong-Wah, M. Yu-Fei, and Z. Hong-Jiang. Video summarization and scene detection by graph modeling.Circuits and Systems for Video Technology, IEEE Transactions on, 15(2):296--305, 2005.
[5]
B. Feng, P. Ding, J. Chen, J. Bai, S. Xu, and B. Xu. Multi-modal information fusion for news story segmentation in broadcast video. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, pages 1417--1420, 2012.
[6]
S. Jianbo and J. Malik. Normalized cuts and image segmentation.Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(8):888--905, 2000.
[7]
Y. Jinhui, W. Huiyi, X. Lan, Z. Wujie, L. Jianmin, L. Fuzong, and Z. Bo. A formal study of shot boundary detection.Circuits and Systems for Video Technology, IEEE Transactions on, 17(2):168--186, 2007.
[8]
R. Klinger, K. Tomanek, and R. Klinger. Classical probabilistic models and conditional random Fields, 2007.
[9]
T. Kudo. Crf++: Yet another crf toolkit, 2005.
[10]
H. Lee, J. Yu, Y. Im, J.-M. Gil, and D. Park. A unified scheme of shot boundary detection and anchor shot detection in news video story parsing.Multimedia Tools and Applications, 51(3):1127--1145, 2011.
[11]
Y. Li and C. Dorai. Svm-based audio classification for instructional video analysis. In Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on, volume 5, pages 897--900, 2004.
[12]
C. Petersohn. Logical unit and scene detection: a comparative survey. InMultimedia Content Access: Algorithms and Systems II, volume 6820, pages 02--17, 2008.
[13]
Z. Rasheed and M. Shah. Detection and representation of scenes in videos.Multimedia, IEEE Transactions on, 7(6):1097--1105, 2005.
[14]
U. Sakarya and Z. Telatar. Video scene detection using graph-based representations. Signal Processing: Image Communication, 25(10):774--783, 2010.
[15]
C. Sutton and A. McCallum. An Introduction to Conditional Random Fields.ArXiv e-prints, 2010.
[16]
J. Wang, L. Duan, Q. Liu, H. Lu, and J. Jin. A multimodal scheme for program segmentation and representation in broadcast video streams. Multimedia, IEEE Transactions on, 10(3):393--408, 2008.
[17]
G. Xinbo and T. Xiaoou. Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing. Circuits and Systems for Video Technology, IEEE Transactions on, 12(9):765--776, 2002.
[18]
M. Yeung, B.-L. Yeo, and B. Liu. Segmentation of video by clustering and graph analysis. Computer Vision and Image Understanding, 71(1):94--109, 1998.
[19]
Z. Yu, Z. Hongjiang, and A. K. Jain. Automatic caption localization in compressed video. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(4):385--392, 2000.
[20]
Z. Yun and M. Shah. Video scene segmentation using markov chain monte carlo.Multimedia, IEEE Transactions on, 8(4):686--697, 2006.

Cited By

View all
  • (2022)Only overlay text: novel features for TV news broadcast video segmentationMultimedia Tools and Applications10.1007/s11042-022-12917-w81:21(30493-30517)Online publication date: 1-Sep-2022
  • (2019)Segmenting with style: detecting program and story boundaries in TV news broadcast videosMultimedia Tools and Applications10.1007/s11042-019-7699-978:22(31925-31957)Online publication date: 27-Jul-2019
  • (2019)A system for semantic segmentation of TV news broadcast videosMultimedia Tools and Applications10.1007/s11042-019-08445-979:9-10(6191-6225)Online publication date: 13-Dec-2019
  • Show More Cited By

Index Terms

  1. A general Framework of video segmentation to logical unit based on conditional random fields

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICMR '13: Proceedings of the 3rd ACM conference on International conference on multimedia retrieval
    April 2013
    362 pages
    ISBN:9781450320337
    DOI:10.1145/2461466
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 April 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. conditional random field
    2. scene segmentation
    3. topic unit segmentation

    Qualifiers

    • Poster

    Conference

    ICMR'13
    Sponsor:

    Acceptance Rates

    ICMR '13 Paper Acceptance Rate 38 of 96 submissions, 40%;
    Overall Acceptance Rate 254 of 830 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Only overlay text: novel features for TV news broadcast video segmentationMultimedia Tools and Applications10.1007/s11042-022-12917-w81:21(30493-30517)Online publication date: 1-Sep-2022
    • (2019)Segmenting with style: detecting program and story boundaries in TV news broadcast videosMultimedia Tools and Applications10.1007/s11042-019-7699-978:22(31925-31957)Online publication date: 27-Jul-2019
    • (2019)A system for semantic segmentation of TV news broadcast videosMultimedia Tools and Applications10.1007/s11042-019-08445-979:9-10(6191-6225)Online publication date: 13-Dec-2019
    • (2016)News Program Detection in TV Broadcast VideosProceedings of the 24th ACM international conference on Multimedia10.1145/2964284.2967281(546-550)Online publication date: 1-Oct-2016
    • (2016)Story segmentation in TV news broadcast2016 23rd International Conference on Pattern Recognition (ICPR)10.1109/ICPR.2016.7900085(2948-2953)Online publication date: Dec-2016
    • (2014)Multiple style exploration for story unit segmentation of broadcast news videoMultimedia Systems10.1007/s00530-013-0350-020:4(347-361)Online publication date: 1-Jul-2014
    • (2014)Anchor Shot Detection with Deep Neural NetworkProceedings of the 15th Pacific-Rim Conference on Advances in Multimedia Information Processing --- PCM 2014 - Volume 887910.1007/978-3-319-13168-9_34(304-312)Online publication date: 1-Dec-2014

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media