Regular Article
A New Method for Region-Based Depth Ordering in a Video Sequence: Application to Frame Interpolation

https://doi.org/10.1006/jvci.2001.0490Get rights and content

Abstract

Region-based interpolation schemes for video sequences have grown in popularity with the introduction of the new object-based representation of video such as in the MPEG4 standard. A new method of a region-based interpolation of missing frames in video sequences for a rate up-conversion is proposed. The method allows an interpolation of both original video frames and segmentation maps for object-based content manipulation. The method uses a 212D model of video scene content corresponding to the depth ordering of plane motion-homogeneous regions along the depth axis. A method for determining the depth order of regions in a frame is developed based on a motion analysis in occluding areas and an optimal graph search. This depth order is used for a motion-compensated interpolation based on an affine motion model. Results in real video sequences in CCIR601 format for up-conversion of video rate are obtained, showing a good quality compared to a B-frame interpolation scheme of MPEG 1,2 standards.

References (21)

  • R. Thoma et al.

    Motion compensating interpolation considering covered and uncovered background

    Signal Process. Image Commun.

    (1989)
  • J. Benois-Pineau et al.

    Hierarchical segmentation of video sequences for content manipulation and adaptive coding

    Signal Process.

    (1998)
  • T. Bonse

    Visually adapted temporal subsampling of motion information

    Signal Process. Image Commun.

    (1994)
  • R. Szieliski and P. Anandan, From 2D Images to 2,5D sprites: A layered approach to modelling 3D scenes, in Proceedings,...
  • M. Pardas, Relative Depth Estimation and segmentation in monocular sequences, in Proceedings, PCS'97, Berlin, Sept....
  • ISO/IEC 14496-1 Information Technology—Coding of audio-visual objects, Part 1,2: JTC 1/SC 29/WG 11 N 2501 (MPEG4...
  • T. T. Chao and C. L. Huang, Motion-compensated spatio-temporal interpolation for frame rate up-conversion of interlaced...
  • A. Pelagotti and G. de Haan, High quality video on MultiMedia PCs, in Proceedings, IEEE Multimedia Systems'99,...
  • F. Marqués et al.

    Prediction of image partitions using Fourier descriptors: Application to segmentation-based coding schemes

    IEEE Trans. Image Process.

    (1998)
  • R. Lancini, A moving object identification algorithm for image sequence interpolation, in Proceedings, IEEE ICIP'98,...
There are more references available in the full text version of this article.

Cited by (21)

View all citing articles on Scopus
View full text