Skip to main content
SpringerLink
Log in

Optical flow approximation based motion object extraction for MPEG-2 video stream

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a compressed-domain motion object extraction algorithm based on optical flow approximation for MPEG-2 video stream. The discrete cosine transform (DCT) coefficients of P and B frames are estimated to reconstruct DC + 2AC image using their motion vectors and the DCT coefficients in I frames, which can be directly extracted from MPEG-2 compressed domain. Initial optical flow is estimated with Black’s optical flow estimation framework, in which DC image is substituted by DC + 2AC image to provide more intensity information. A high confidence measure is exploited to generate dense and accurate motion vector field by removing noisy and false motion vectors. Global motion estimation and iterative rejection are further utilized to separate foreground and background motion vectors. Region growing with automatic seed selection is performed to extract accurate object boundary by motion consistency model. The object boundary is further refined by partially decoding the boundary blocks to improve the accuracy. Experimental results on several test sequences demonstrate that the proposed approach can achieve compressed-domain video object extraction for MPEG-2 video stream in CIF format with real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Zhang, D.S., Lu, G.J.: Segmentation of moving objects in image sequence: a review. Circuit. Syst. Signal Process. 20(2), 143–183 (2001). doi:10.1007/BF01201137

    Article  MATH  Google Scholar 

  2. Daras, P., Kompatsiaris, I., Grinias, I., Akrivas, G., Kollias, S., Strintzis, M.: MPEG-4 authoring tool using moving object segmentation and tracking in video shots. EURASIP J. Appl. Signal Process. 9(2), 1–18 (2003)

    Google Scholar 

  3. Luo, H.T., Eleftheriadis, A.: Model-based segmentation and tracking of head-and-shoulder video objects for real time multimedia services. IEEE Trans. Multimed. 5(3), 379–389 (2003). doi:10.1109/TMM.2003.813285

    Article  Google Scholar 

  4. Manerba, F., Benois-Pineau, J., Leonardi, R., Mansencal, B.: Multiple moving object detection for fast video content description in compressed domain. EURASIP J. Adv. Signal Process. 231930, 1–15 (2008). doi:10.1155/2008/231930

    Article  Google Scholar 

  5. Mezaris, V., Kompatsiaris, I., Boulgouris, N.V., Strintzis, M.G.: Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval. IEEE Trans. Circuit Syst. Video Technol. 14(5), 606–621 (2004). doi:10.1109/TCSVT.2004.826768

    Article  Google Scholar 

  6. Liu, Z., Lu, Y., Zhang, Z.Y.: Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain. J. Vis. Commun. Image Represent. 18(3), 275–290 (2007). doi:10.1016/j.jvcir.2007.02.002

    Article  Google Scholar 

  7. Sukmarg, O., Rao, K.R.: Fast object detection and segmentation in MPEG compressed domain. In: Proc. IEEE TENCON, vol. 2, pp. 364–368 (2000)

  8. Babu, R.V., Ramakrishnan, K.R., Srinivasan, S.H.: Video object segmentation: a compressed domain approach. IEEE Trans. Circuit Syst. Video Tech. 14(4), 462–674 (2004). doi:10.1109/TCSVT.2004.825536

    Article  Google Scholar 

  9. Porikli, F.: Real-time video object segmentation for MPEG encoded video sequences. In: Proc. SPIE Conf. Real Time Imaging, vol. 5297, pp. 195–203 (2004)

  10. Zeng, W., Du, J., Gao, W., Huang, Q.M.: Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model. Real-Time Imaging 11, 290–299 (2005). doi:10.1016/j.rti.2005.04.008

    Article  Google Scholar 

  11. Coimbra, M.T., Davies, M.: Approximating optical flow within the MPEG-2 compressed domain. IEEE Trans. Circuit Syst. Video Tech. 15(1), 103–107 (2005). doi:10.1109/TCSVT.2004.837016

    Article  Google Scholar 

  12. You, J.Y., Liu, G.Z., Li, H.L.: A fast and robust optical flow estimation method for compressed video. J. Electron. Inf. Technol. 29(9), 2154–2157 (2007). (in Chinese)

    Google Scholar 

  13. Chen, D.B., Schultz, R.R.: Extraction of high-resolution video stills from MPEG image sequences. Proc. IEEE ICIP 2, 465–469 (1998)

    Google Scholar 

  14. Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow field. Comput. Vis. Image Underst. 63(1), 75–104 (1996). doi:10.1006/cviu.1996.0006

    Article  Google Scholar 

  15. Rapantzikos, K.E.: Dense Estimation of optical flow in the compressed domain using robust techniques, http://www.image.ece.ntua.gr/~rap/ (2002)

  16. Yeo, B.L., Liu, B.: On the extraction of DC sequence from MPEG compressed video. Proc. IEEE ICIP 2, 260–263 (1995)

    Google Scholar 

  17. Kobla, V., Doermann, D., Lin, K.I., Christos, F.: Compressed domain video indexing techniques using DCT and motion vector information in MPEG video. Proc. SPIE Storage Retr. Image Video Database 3022, 200–210 (1997)

    Google Scholar 

  18. Simoncelli E.P, Adelson E.H., Heeger D.J. (1991) Probability distribution of optical flow. IEEE Proc. Comput. Vis. Pattern Recognit. 310–315

  19. Rath, G.B., Makur, A.: Iterative least squares and compression based estimation for a four-parameter linear global motion model and global motion compensation. IEEE Trans. Circ. Syst. Video Technol. 9(7), 1075–1099 (1999). doi:10.1109/76.795060

    Article  Google Scholar 

  20. Chung, R.H., Chin, F.Y., Wong, K.Y., Chow, K.P., Luo, T., Fung, H.S.: Efficient block-based motion segmentation method using motion vector consistency. In: Proceedings of IAPR Conference on Machine Vision Applications (MVA2005), pp. 550–553 (2005)

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant 60572127 and Hunan Provincial Natural Science Foundation under Grant 05JJ30113. The authors greatly appreciate the anonymous reviewers for their constructive comments, and Mr. Alex Asiimwe for his help to improve the English usage.

Author information

Authors and Affiliations

  1. Room 504, College of Computer and Communication, Hunan University, Lushan Road, 410082, Changsha, Hunan, China

    Gaobo Yang, Weiwei Chen & Qiya Zhou

  2. Key Lab of Advanced Display and System Integration (Ministry of Education), Shanghai University, 200072, Shanghai, China

    Zhaoyang Zhang

Authors
  1. Gaobo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiya Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhaoyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaobo Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, G., Chen, W., Zhou, Q. et al. Optical flow approximation based motion object extraction for MPEG-2 video stream. J Real-Time Image Proc 4, 303–316 (2009). https://doi.org/10.1007/s11554-009-0113-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-009-0113-5

Keywords

Search

Navigation