Abstract
Object-oriented motion segmentation is a basic step of the effective coding of image-series. Following the MPEG-4 standard we should define such objects. In this paper, a fully parallel and locally connected computation model is described for segmenting frames of image sequences based on spatial and motion information. The first type of the algorithm is called early segmentation. It is based on spatial information only and aims at providing an over-segmentation of the frame in real-time. Even if the obtained results do not minimize the number of regions, it is a good starting point for higher level post processing, when the decision on how to regroup regions in object can rely on both spatial and temporal information. In the second type of the algorithm stochastic optimization methods are used to form homogenous dense optical vector fields which act directly on motion vectors instead of 2D or 3D motion parameters. This makes the algorithm simple and less time consuming than many other relaxation methods. Then we apply morphological operators to handle disocclusion effects and to map the motion field to the spatial content. Computer simulations of the CNN architecture demonstrate the usefulness of our methods. All solutions in our approach suggest a fully parallel implementation in a newly developed CNN-UM VLSI chip architecture.
Similar content being viewed by others
References
T. Ebrahimi et al., “Dynamic coding of visual information,” Technical description ISO/IEC JTC1/SC2/WG11/M0320, MPEG-4, Swiss Federal Institute of Technology, Oct. 1995.
N. Pal and S. Pal, “A review on image segmentation techniques,” Pattern Recognition, Vol. 26, No. 9, pp. 1277–1294, 1993.
R. Haralick, “Image segmentation survey,” Fundamentals in Computer Vision, Cambridge University Press, 1983.
P. Bouthemy and E. Francois, “Motion segmentation and qualitative dynamic scene analysis from an image sequence,” International Journal of Computer Vision, Vol. 10, No. 2, pp. 157–182, 1993.
Klaus Illgner and Frank Müller, “Image segmentation using motion estimation,” Time-Varying Image Processing and Moving Object Recognition, V. Cappellini (Ed.), Elsevier Science B.V., Amsterdam, Vol. 4, pp. 238–243, 1997.
F. Moscheni, S. Bhattacharjee, and M. Kunt, “Spatiotemporal segmentation based on region merging,” IEEE Transactions PAMI, Vol. 20, No. 9, pp. 897–915, 1998.
P.A. Laplante and A.D. Stoyenko (Ed.), Real-Time Imaging, Theory, Techniques, and Applications, IEEE Press, 1996.
R. Domínguez-Castro, S. Espejo, A. Rodríguez-Vázquez, A. Carmona, P. Földesy, Á. Zarándy, P Szolgay, T. Szirányi, and T. Roska, “A 0.8µm CMOS two-dimensional programmable mixed-signal focal-plane array processor with onchip binary imaging and instructions storage,” IEEE Journal of Solid-State Circuits, Vol. 32, No. 7, pp. 1013–1026.
T. Roska and L.O. Chua. "The CNN universal machine: An analogic array computer,” IEEE Transactions on Circuits and Systems-II, Vol. 40, pp. 163–173, March 1993.
T. Szirányi and M. Csapodi, “Texture classification and segmentation by cellular neural network using genetic learning,” Computer Vision and Image Understanding, Vol. 71, No. 3, pp. 255–270, Sep. 1998.
T. Toyoda, Y. Nitta, E. Funatsu, Y. Miyake, W. Freeman, J. Ohta, and K. Kyuma, “Artificial retina chips as image input interfaces for multimedia systems,” Proceedings of the Optoelectronics and Communications Conference, OECC'96, Chiba, Japan, July 1996.
T. Aach, A. Kaup, and R. Mester, “Statistical model-based change detection in moving video,” Signal Processing, Vol. 31, pp. 165–180, 1993.
P. Perona, T. Shiota, and J. Malik, Anisotropic Diffusion, Geometry Driven Diffusion In Computer Vision, Kluwer Academic Publishers, pp. 73–92, 1992.
S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 6, pp. 721–741, 1984.
T. Szirányi and J. Zerubia, “Markov random field image segmentation using cellular neural network,” IEEE Transactions on Circuits and Systems I., Vol. 44, pp. 86–89, Jan. 1997.
T. Szirányi, J. Zerubia, L. CzÚni, D. Geldreich, and Z. Kato, “Image segmentation using markov random field model in fully parallel cellular network architectures,” Real-Time Imaging, Acad. Press, August 2000, in press.
T. Szirányi and L. CzÚni, “Image compression by orthogonal decomposition using cellular neural network chips,” Int. J. Circuit Theory and Applications, Vol. 27, No. 1, pp. 117–134, 1999.
T. Kozek, Á Zarándy, S. Zöld, T. Roska, and P. Szolgay. "Analogic macro code (AMC)-extended assembly language for CNN computers,” Report MTA SZTAKI, Budapest, 1998.
T. Roska, L. Kék, L. Nemes, Á. Zarándy, M. Brendel, and P. Szolgay, CNN Software Library (Templates and Algorithms), Version 7.2, DNS-1-1998, (CADET-15), Computer and Automation Institute, Hungarian Academy of Sciences, Budapest, 1998.
P.L. Venetianer, F. Werblin, T. Roska, and L.O. Chua, “Analogic CNN algorithms for some image compression and restoration tasks,” IEEE Trans. on Circuits and Systems I: Fundamental Theory and Applications, (CAS-I), Vol. 42, pp. 278–284, May 1995.
K. Laszlo, F. Ziliani, T. Roska, and M. Kunt, “Early segmentation in video compression using CNN processors,” Proceedings of the 1998 Fifth International Workshop on Cellular Neural Networks and Theirs Applications (CNNA'98), London, UK, pp. 175–180, April 14-17, 1998.
S. Fejes and L.S. Davis, “What can projections of flow fields tell us about the visual motion,” Proceedings of the ICCV, Bombay, India, 1998.
J.L. Barron, D.J. Fleet, and S. Beauchemin, “Performance of optical flow techniques,” International Journal of Computer Vision, Vol. 12, No. 1, pp. 43–77, 1994.
B.E. Shi, T. Roska, and L.O. Chua, “Estimating optical flow with cellular neural networks,” International Journal of Circuit Theory and Applications, Vol. 26, No. 4, pp. 343–364, July 1998.
J.N. Pan, Y.Q. Shi, and C.Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Transactions on Image Processing, Vol. 7, pp. 1061–1067, July 1998.
D.J. Fleet and K. Langley, “Recursive filters for optical Flow,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, pp. 61–67, 1995.
N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, “Equation of state calculations by fast computing machines,” J. of Chemical Physics, Vol. 21, No. 6, pp. 1087–1092, 1953.
L. Alvarez, F. Guichard, P.L. Lions, and J.M. Morel, Axioms and Fundamental Equations of Image Processing, Ceremade, France, 1993.
T. Roska and T. Szirányi, “Classes of analogic CNN algorithms and their practical use in complex image processing tasks,” Proceedings of the IEEE Nonlinear Signal and Image Processing Conf., pp. 767–770, 1995.
Cs. Rekeczky, T. Roska, and A. Ushida, “CNN based difference-controlled adaptive nonlinear image filters,” Int. J. Circuit Theory and Applications, Vol. 26, pp. 375–423, 1998.
T. Szirányi, I. Kopilovic, and B.P. Tóth, “Anisotropic diffusion as a preprocessing step for efficient image compression,” Proceedings of the 14th ICPR, Brisbane, IAPR, Australia, pp. 1565–1567, August 16-20, 1998.
Á. Zarándy, A. Stoffels, T. Roska, and L.O. Chua, “Implementation of binary and gray-scale mathematical morphology on the CNNuniversal machine,” IEEE Trans. on Circuits and Systems I: Fundamental Theory and Applications, (CAS-I), Vol. 45, No. 2, pp. 163–168, 1998.
L. CzÚni, T. Szirányi, and J. Zerubia, “Multigrid MRF based picture segmentation with cellular neural networks,” CAIP'97, Kiel, Proceedings in Lecture Notes in Computer Science, Vol. 1296, pp. 345–352, 1997.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Szirányi, T., László, K., Czúni, L. et al. Object Oriented Motion-Segmentation for Video-Compression in the CNN-UM. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 23, 479–496 (1999). https://doi.org/10.1023/A:1008117724074
Published:
Issue Date:
DOI: https://doi.org/10.1023/A:1008117724074