Skip to main content
SpringerLink
Log in

Object Oriented Motion-Segmentation for Video-Compression in the CNN-UM

  • Published:
Journal of VLSI signal processing systems for signal, image and video technology Aims and scope Submit manuscript

Abstract

Object-oriented motion segmentation is a basic step of the effective coding of image-series. Following the MPEG-4 standard we should define such objects. In this paper, a fully parallel and locally connected computation model is described for segmenting frames of image sequences based on spatial and motion information. The first type of the algorithm is called early segmentation. It is based on spatial information only and aims at providing an over-segmentation of the frame in real-time. Even if the obtained results do not minimize the number of regions, it is a good starting point for higher level post processing, when the decision on how to regroup regions in object can rely on both spatial and temporal information. In the second type of the algorithm stochastic optimization methods are used to form homogenous dense optical vector fields which act directly on motion vectors instead of 2D or 3D motion parameters. This makes the algorithm simple and less time consuming than many other relaxation methods. Then we apply morphological operators to handle disocclusion effects and to map the motion field to the spatial content. Computer simulations of the CNN architecture demonstrate the usefulness of our methods. All solutions in our approach suggest a fully parallel implementation in a newly developed CNN-UM VLSI chip architecture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. T. Ebrahimi et al., “Dynamic coding of visual information,” Technical description ISO/IEC JTC1/SC2/WG11/M0320, MPEG-4, Swiss Federal Institute of Technology, Oct. 1995.

  2. N. Pal and S. Pal, “A review on image segmentation techniques,” Pattern Recognition, Vol. 26, No. 9, pp. 1277–1294, 1993.

    Article  Google Scholar 

  3. R. Haralick, “Image segmentation survey,” Fundamentals in Computer Vision, Cambridge University Press, 1983.

  4. P. Bouthemy and E. Francois, “Motion segmentation and qualitative dynamic scene analysis from an image sequence,” International Journal of Computer Vision, Vol. 10, No. 2, pp. 157–182, 1993.

    Article  Google Scholar 

  5. Klaus Illgner and Frank Müller, “Image segmentation using motion estimation,” Time-Varying Image Processing and Moving Object Recognition, V. Cappellini (Ed.), Elsevier Science B.V., Amsterdam, Vol. 4, pp. 238–243, 1997.

    Chapter  Google Scholar 

  6. F. Moscheni, S. Bhattacharjee, and M. Kunt, “Spatiotemporal segmentation based on region merging,” IEEE Transactions PAMI, Vol. 20, No. 9, pp. 897–915, 1998.

    Article  Google Scholar 

  7. P.A. Laplante and A.D. Stoyenko (Ed.), Real-Time Imaging, Theory, Techniques, and Applications, IEEE Press, 1996.

  8. R. Domínguez-Castro, S. Espejo, A. Rodríguez-Vázquez, A. Carmona, P. Földesy, Á. Zarándy, P Szolgay, T. Szirányi, and T. Roska, “A 0.8µm CMOS two-dimensional programmable mixed-signal focal-plane array processor with onchip binary imaging and instructions storage,” IEEE Journal of Solid-State Circuits, Vol. 32, No. 7, pp. 1013–1026.

  9. T. Roska and L.O. Chua. "The CNN universal machine: An analogic array computer,” IEEE Transactions on Circuits and Systems-II, Vol. 40, pp. 163–173, March 1993.

    Article  MathSciNet  MATH  Google Scholar 

  10. T. Szirányi and M. Csapodi, “Texture classification and segmentation by cellular neural network using genetic learning,” Computer Vision and Image Understanding, Vol. 71, No. 3, pp. 255–270, Sep. 1998.

    Article  Google Scholar 

  11. T. Toyoda, Y. Nitta, E. Funatsu, Y. Miyake, W. Freeman, J. Ohta, and K. Kyuma, “Artificial retina chips as image input interfaces for multimedia systems,” Proceedings of the Optoelectronics and Communications Conference, OECC'96, Chiba, Japan, July 1996.

  12. T. Aach, A. Kaup, and R. Mester, “Statistical model-based change detection in moving video,” Signal Processing, Vol. 31, pp. 165–180, 1993.

    Article  MATH  Google Scholar 

  13. P. Perona, T. Shiota, and J. Malik, Anisotropic Diffusion, Geometry Driven Diffusion In Computer Vision, Kluwer Academic Publishers, pp. 73–92, 1992.

  14. S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 6, pp. 721–741, 1984.

    Article  MATH  Google Scholar 

  15. T. Szirányi and J. Zerubia, “Markov random field image segmentation using cellular neural network,” IEEE Transactions on Circuits and Systems I., Vol. 44, pp. 86–89, Jan. 1997.

    Article  Google Scholar 

  16. T. Szirányi, J. Zerubia, L. CzÚni, D. Geldreich, and Z. Kato, “Image segmentation using markov random field model in fully parallel cellular network architectures,” Real-Time Imaging, Acad. Press, August 2000, in press.

  17. T. Szirányi and L. CzÚni, “Image compression by orthogonal decomposition using cellular neural network chips,” Int. J. Circuit Theory and Applications, Vol. 27, No. 1, pp. 117–134, 1999.

    Article  Google Scholar 

  18. T. Kozek, Á Zarándy, S. Zöld, T. Roska, and P. Szolgay. "Analogic macro code (AMC)-extended assembly language for CNN computers,” Report MTA SZTAKI, Budapest, 1998.

  19. T. Roska, L. Kék, L. Nemes, Á. Zarándy, M. Brendel, and P. Szolgay, CNN Software Library (Templates and Algorithms), Version 7.2, DNS-1-1998, (CADET-15), Computer and Automation Institute, Hungarian Academy of Sciences, Budapest, 1998.

    Google Scholar 

  20. P.L. Venetianer, F. Werblin, T. Roska, and L.O. Chua, “Analogic CNN algorithms for some image compression and restoration tasks,” IEEE Trans. on Circuits and Systems I: Fundamental Theory and Applications, (CAS-I), Vol. 42, pp. 278–284, May 1995.

    Article  Google Scholar 

  21. K. Laszlo, F. Ziliani, T. Roska, and M. Kunt, “Early segmentation in video compression using CNN processors,” Proceedings of the 1998 Fifth International Workshop on Cellular Neural Networks and Theirs Applications (CNNA'98), London, UK, pp. 175–180, April 14-17, 1998.

  22. S. Fejes and L.S. Davis, “What can projections of flow fields tell us about the visual motion,” Proceedings of the ICCV, Bombay, India, 1998.

  23. J.L. Barron, D.J. Fleet, and S. Beauchemin, “Performance of optical flow techniques,” International Journal of Computer Vision, Vol. 12, No. 1, pp. 43–77, 1994.

    Article  Google Scholar 

  24. B.E. Shi, T. Roska, and L.O. Chua, “Estimating optical flow with cellular neural networks,” International Journal of Circuit Theory and Applications, Vol. 26, No. 4, pp. 343–364, July 1998.

    Article  MATH  Google Scholar 

  25. J.N. Pan, Y.Q. Shi, and C.Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Transactions on Image Processing, Vol. 7, pp. 1061–1067, July 1998.

    Article  Google Scholar 

  26. D.J. Fleet and K. Langley, “Recursive filters for optical Flow,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, pp. 61–67, 1995.

    Article  Google Scholar 

  27. N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, “Equation of state calculations by fast computing machines,” J. of Chemical Physics, Vol. 21, No. 6, pp. 1087–1092, 1953.

    Article  Google Scholar 

  28. L. Alvarez, F. Guichard, P.L. Lions, and J.M. Morel, Axioms and Fundamental Equations of Image Processing, Ceremade, France, 1993.

  29. T. Roska and T. Szirányi, “Classes of analogic CNN algorithms and their practical use in complex image processing tasks,” Proceedings of the IEEE Nonlinear Signal and Image Processing Conf., pp. 767–770, 1995.

  30. Cs. Rekeczky, T. Roska, and A. Ushida, “CNN based difference-controlled adaptive nonlinear image filters,” Int. J. Circuit Theory and Applications, Vol. 26, pp. 375–423, 1998.

    Article  MATH  Google Scholar 

  31. T. Szirányi, I. Kopilovic, and B.P. Tóth, “Anisotropic diffusion as a preprocessing step for efficient image compression,” Proceedings of the 14th ICPR, Brisbane, IAPR, Australia, pp. 1565–1567, August 16-20, 1998.

  32. Á. Zarándy, A. Stoffels, T. Roska, and L.O. Chua, “Implementation of binary and gray-scale mathematical morphology on the CNNuniversal machine,” IEEE Trans. on Circuits and Systems I: Fundamental Theory and Applications, (CAS-I), Vol. 45, No. 2, pp. 163–168, 1998.

    Article  Google Scholar 

  33. L. CzÚni, T. Szirányi, and J. Zerubia, “Multigrid MRF based picture segmentation with cellular neural networks,” CAIP'97, Kiel, Proceedings in Lecture Notes in Computer Science, Vol. 1296, pp. 345–352, 1997.

Download references

Author information

Authors and Affiliations

  1. Analogical and Neural Computing Laboratory, Comp. & Automation Inst., Hungarian Academy of Sciences, H-1111, Budapest, Kende u. 13-17, Hungary

    Tamás Szirányi & Károly László

  2. Department of Image Processing and Neurocomputing, University of Veszprém, H-8200, Veszprém, Egyetem u. 10, Hungary

    László Czúni

  3. Signal Processing Laboratory, Swiss Federal Institute of Technology, CH-1015, Lausanne, Switzerland

    Francesco Ziliani

Authors
  1. Tamás Szirányi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Károly László
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. László Czúni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Ziliani
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Szirányi, T., László, K., Czúni, L. et al. Object Oriented Motion-Segmentation for Video-Compression in the CNN-UM. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 23, 479–496 (1999). https://doi.org/10.1023/A:1008117724074

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008117724074

Keywords

Search

Navigation