Regular Article
Automatic Moving Object Extraction toward Content-Based Video Representation and Indexing

https://doi.org/10.1006/jvci.2001.0471Get rights and content

Abstract

In this paper, we present a novel semantic video object generation and temporal tracking technique for providing content-based video representation and indexing. In our system, the homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and similarity-based region growing procedures. Then the semantic objects of meaning to human users are generated by an automatic seeded region aggregation or human interaction procedure. These generated semantic objects are then tracked along the time axis for exploiting their temporal correspondences among frames. The video sources in databases can be accessed through these individual video objects obtained and they are represented by a set of visual features, such as its trajectory, set of boundary pixels, area of the object region, shape, color histogram, orientation, and regularity. A learning-based optimization procedure is used for selecting the suitable dimensional weighting coefficients, so that the feature-based object similarity corresponds to concept-based similarity directly. Moreover, a seeded semantic video content clustering technique for providing cluster-based hierarchical video indexing, retrieval, and browsing is proposed.

References (61)

  • J.-Y. Chen et al.

    ViBE: A compressed video database structured for active browsing and search

    Proc. SPIE: Multimedia Storage and Archiving Systems IV

    (1999)
  • A. Humrapur, A. Gupta, B. Horowitz, C. F. Shu, C. Fuller, J. Bach, M. Gorkani, and R. Jain, Virage video engine, in...
  • A.K. Jain et al.

    Query by video clip

    ACM Multimedia Systems

    (1999)
  • Y. Rui et al.

    Constructing table-of-content for videos

    ACM Multimedia Systems

    (1999)
  • M. M. Yeung, B.-L. Yeo, and, B. Liu, Extracting story units from long program for video browsing and navigation, in,...
  • M. Irani et al.

    Video indexing based on mosaic representation

    Proc. IEEE

    (1998)
  • J. Fan

    Adaptive motion-compensated video coding scheme towards content-based bit rate allocation

    J. Electr. Imaging

    (2000)
  • E. Oomoto et al.

    OVID: Design and implementation of a video object database system

    IEEE Trans. Knowledge Data Eng.

    (1993)
  • J. R. Smith and S. F. Chang, VisualSEEK: A fully automated content-based image query system, in ACM Multimedia Conf....
  • S.F. Chang et al.

    A fully automatic content-based video search engine supporting spatiotemporal queries

    IEEE Trans. Circuits Systems Video Technol.

    (1998)
  • Y. Deng and B. S. Manjunath, NeTra-V: Toward an object-based video representation, IEEE Trans. Circuits Systems Video...
  • A. Pentland, R. W. Picard, and S. Sclaroff, Photobook: Content-based manipulation of image databases, Int. J. Comput....
  • J. D. Courtney, Automatic video indexing via object motion analysis, Pattern Recognit.30, 1997,...
  • Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra, Relevance feedback: A power tool for interactive content-based image...
  • S. Satoh, and, T. Kanade, Name-It: Association of face and name in video, in, Proc. of Computer Vision and Pattern...
  • G. Adiv

    Determining three-dimensional motion and structure from optical flow generated by several moving objects

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1985)
  • J. Wang et al.

    Representing moving image with layers

    IEEE Trans. Image Process.

    (1994)
  • S. Geman et al.

    Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1984)
  • P. Bouthemy and E. Francois, Motion segmentation and qulitative dynamic scene analysis from an image sequence, Intl. J....
  • C. H. Yang and J. Konrad, Motion-based object segmentation using continuation using MDL priciples, Proc. SPIE:...
  • Cited by (0)

    f1

    [email protected]

    1

    To whom correspondence should be addressed.

    View full text