Skip to main content
SpringerLink
Log in

Extraction of Video Objects via Surface Optimization and Voronoi Order

  • Published:
Journal of VLSI signal processing systems for signal, image and video technology Aims and scope Submit manuscript

Abstract

We implement a video object segmentation system that integrates the novel concept of Voronoi Order with existing surface optimization techniques to support the MPEG-4 functionality of object-addressable video content in the form of video objects. The major enabling technology for the MPEG-4 standard are systems that compute video object segmentation, i.e., the extraction of video objects from a given video sequence. Our surface optimization formulation describes the video object segmentation problem in the form of an energy function that integrates many visual processing techniques. By optimizing this surface, we balance visual information against predictions of models with a priori information and extract video objects from a video sequence. Since the global optimization of such an energy function is still an open problem, we use Voronoi Order to decompose our formulation into a tractable optimization via dynamic programming within an iterative framework. In conclusion, we show the results of the system on the MPEG-4 test sequences, introduce a novel objective measure, and compare results against those that are hand-segmented by the MPEG-4 committee.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Irani and B.R.S. Peleg, “Detecting and Tracking Multiple Moving Objects Using Temporal Integration, ” in Proc. Euro. Conf. Computer Vision, Genova, Italy, 1992, pp. 282–287.

  2. D. Wang, “Unsupervised Video Segmentation Based on Water-sheds and Temporal Tracking, ” IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, no. 5, 1995.

  3. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active Contour Models, ” Int'l Journal of Computer Vision, vol. 1, no. 4, 1988, pp. 321–331.

    Article  Google Scholar 

  4. C. Gum and M.-C. Lee, “Semiautomatic Segmentation and Tracking of Semantic Video Objects, ” IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, no. 5, 1997, pp. 572–584.

    Google Scholar 

  5. F. Leymarie and M.D. Levine, “Tracking Deformable Objects in the Plane Using an Active Contour Model, ” IEEE Trans. on Pattern Anal. and Machine Intelligence, vol. 15, no. 6, 1993, pp. 617–634.

    Article  Google Scholar 

  6. A. Neri, S. Colonnese, G. Russo, and P. Talone, “Automatic Moving Object and Background Separation, ” Signal Processing, vol. 2, no. 66, 1998, pp. 219–232.

    Article  Google Scholar 

  7. Til Aach, Andre Kaup, and Rudolf Mester, “Statistical Model-Based Change Detection in Moving Video, ” Signal Processing, vol. 31, no. 2, 1993, pp. 165–180.

    Article  MATH  Google Scholar 

  8. H. Naseri and J.A. Stuller, “Segmentation and Motion Estimation, ” in Proceedings of ICASSP96, 1996, pp. 1906–1909.

  9. P. Bouthemy and E. Francois, “Motion Segmentation and Qualitative Dynamic Scene Analysis from a Image Sequence, ” Int. J. Computer Vision, vol. 10, no. 2, 1993, pp. 157–182.

    Article  Google Scholar 

  10. T. Meier and K.N. Ngan, “Automatic Segmentation of Moving Objects for Video Object Plan Generation, ” IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, no. 5, 1998, pp. 525–538.

    Article  Google Scholar 

  11. J.G. Choi, S.-W. Lee, and S.-D. Kim, “Video Segmentation Based On Spatial and Temporal Information, ” Proceedings of ICASSP97, vol. 4, 1997, pp. 2661–2664.

    Google Scholar 

  12. J.G. Choi, S.-W. Lee, and S.-D. Kim, “Spatio-Temporal Video Segmentation Using a Joint Similarity Measure, ” IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 2, 1997, pp. 279–286.

    Article  Google Scholar 

  13. M.-J.J. Wang, W.-Y. Wu, L.-K. Huang, and D.-M. Wang, “Corner Detection Using Bending Value, ” Pattern Recognition Letters, no. 16, 1995, pp. 575–583.

  14. Y. Altunbasak, R. Oten, and R.J.P. de Figueiredo, “Simultaneous Object Segmentation, Multiple Object Tracking and Alpha Map Generation, ” in Proceedings of 1997 Int. Conf. on Image Proc., 1997, pp. 69–72.

  15. MPEG Committee, MPEG-4 Requirements Doc., ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Associated Audio MPEG98/W2194, March 1998.

  16. D. Lee, “Medial Axis Transformation of a Planar Shape, ” IEEE Transactions on Pattern Recognition and Machine Intelligence, vol. 4, no. 4, 1982, pp. 363–369.

    Article  MATH  Google Scholar 

  17. I.-J. Lin and S.Y. Kung, “Automatic Video Object Segmentation via Voronoi Ordering and Surface Optimization, ” 1999 IEEE Third Workshop on Multimedia Signal Processing, Oct. 1999.

  18. J. Canny, “AComputational Approach to Edge Detection, ” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, 1986, pp. 679–698.

    Article  Google Scholar 

  19. B.K.P. Horn and B.G. Schunck, “Determining Optical Flow, ” Artificial Intelligence, vol. 17, 1981, pp. 185–204.

  20. S.H. Lai and B.C. Vemuri, “Robust Efficient Algorithms for Optical Flow Computation, ” in International Symposium on Computer Vision, 1995, pp. 455–460.

  21. D.H. Ballard and C.M. Brown, Computer Vision, Prentice-Hall Inc., 1982.

  22. F. Heitz and P. Bouthemy, “Multimodel Estimation of Discontinuous Optical Flow Using Markov Random Fields, ” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 12, 1993.

  23. S.Y. Kung, Y.-T. Lin, and Y.K. Chen, “Motion-Based Segmentation by Principal Singular Vector (PSV) Clustering Method, ” in Proceedings of ICASSP96, May 1996.

  24. R. Bellman and S. Dreyfus, Applied Dynamic Programming, Princeton University Press, 1962.

  25. A. Amini, T. Weymouth, and R. Jain, “Using Dynamic Programming for Solving Variational Problems in Vision, ” IEEE Trans. on Pattern Anal. and Mach. Intelligence, vol. 12, no. 9, 1990, pp. 855–867.

    Article  Google Scholar 

  26. I.-J. Lin and S.Y. Kung, “Circular Viterbi Based Adaptive System for Automatic Video Object Segmentation, ” IEEE Second Workshop on Multimedia Signal Processing, Dec. 1998.

  27. S.Y. Kung, Digital Neural Networks, Prentice-Hall, 1993.

  28. T. Sikora, The MPEG-4 Video Standard Verification Model, IEEE Trans. Circuits Syst. Video Technology, vol. 7, 1997, pp. 19–31.

    Article  Google Scholar 

  29. J.K. Aggarwal and N. Nandhakumar, “On the Computation of Motion from Sequences of Images—A Review, ” Proceedings of the IEEE, vol. 76, no. 8, 1988, pp. 917–935

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hewlett-Packard Laboratories, Palo Alto, CA, USA

    I-Jong Lin

  2. Department of Electrical Engineering, Princeton University, Princeton, NJ, 08544-5263

    S.Y. Kung

Authors
  1. I-Jong Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S.Y. Kung
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lin, IJ., Kung, S. Extraction of Video Objects via Surface Optimization and Voronoi Order. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 29, 23–39 (2001). https://doi.org/10.1023/A:1011167329792

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011167329792

Search

Navigation