Abstract
Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind etc. We present a characterization of dynamic textures that poses the problems of modeling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system identification to capture the “essence” of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of second-order stationary processes, we identify the model sub-optimally in closed-form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to infinite length with negligible computational cost. We present experimental evidence that, within our framework, even low-dimensional models can capture very complex visual phenomena.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amari, S. and Cardoso, J. 1997. Blind source separation-semiparametric statistical approach. IEEE Transactions on Signal Processing, 45:692–700.
Arun, K.S. and Kung, S.Y. 1990. Balanced approximation of stochastic systems. SIAM Matrix Analysis and Applications, 42– 68.
Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., and Werman, M. 2001. Texture mixing and texture movie synthesis using statistical learning. IEEE Transactions on Visualization and Computer Graphics, 7(2):120–135.
Barzel, R. 1992. Physically-Based Modeling for Computer Graphics: A Structured Approach. Academic Press, San Diego, CA.
Bauer, D., Deistler, M., and Scherrer, W. 1999. Consistency and asymptotic normality of some subspace algorithms for systems without observed inputs. Automatica, 35(7):1243–1254.
Bigun, J. and du Buf, J.M. 1994. N-folded symmetries by complex moments in gabor space and their application to unsupervised texture segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1):80–87.
Bissacco, A., Chiuso, A., Ma, Y., and Soatto, S. 2001. Recognition of human gaits. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HA.
Blake, A. and Isard, M. 1998. Active Contours. Springer Verlag: Berlin.
Chiuso, A. and Picci, G. To appear. Asymptotic variance of subspace techniques. In Journal of Econometrics.
Cross, G. and Jain, A. 1983. Markov random field texture models.IEEE Transactions on Pattern Analysis and Machine Intelligence, 5:25–40.
de Bonet, J. and Viola, P. 1998. Multiresolution sampling procedure for analysis and synthesis of texture images. In Proceedings IEEE Conf. Computer Vision and Pattern Recognition.
Dempster, A.P., Laird, N.M., and Rubin, D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm J. R. Statist. Soc. B, 39:185–197.
Doretto, G. and Soatto, S. 2002. Editable dynamic textures. Technical Report 020001, UCLA Computer Science Department.
Ebert, D., Carlson, W., and Parent, R. 1994. Solid spaces and inverse particle systems for controlling the animation of gases and fluids. The Visual Computer, 10(4):179–190.
Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Seventh International Conference on Computer Vision, Corfu, Greece.
Fitzgibbon, A.W. 2001. Stochastic rigidity: Image registration for nowhere-static scenes. In Proc. of the Intl. Conf. on Comp. Vis., pp. 662–670.
Fournier, A. and Reeves, W. 1986. A simple model of ocean waves. In ACM SIGGRAPH Proceedings, pp. 75–84.
Freeman, W. and Adelson, E. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9):891–906.
Golub, G. and Van Loan, C. 1989. Matrix Computations, 2nd edn. Johns Hopkins University Press: Baltimore, MD.
Hassner, M. and Sklansky, J. 1981. The use of markov random fields as models of texture. In Image Modeling. Academic Press: San Diego, CA.
Heeger, D. and Bergen, J. 1995. Pyramid-based texture analysis /synthesis. In ACM SIGGRAPH Conference Proceedings.
Hodgins, J.K. and Wooten, W.L. 1998. Animating human athletes. In Robotics Research: The Eighth International Symposium, Y. Shirai and S. Hirose (Eds.). Springer-Verlag: Berlin, Germany, pp. 356–367.
Hunt, R.W. 1996. The Reproduction of Colour, 5th edn. Fisher Books.
Julesz, B. 1962. Visual pattern discrimination. IRE Trans. Info Theory, IT-8.
Kailath, T. 1980. Linear Systems. Prentice Hall: Englewood Cliffs, NJ.
Kirsch, A. 1996. An Introduction to the Mathematical Theory of Inverse Problems. Springer-Verlag: New York.
Lindquist, A. and Picci, G. 1979. The stochastic realization problem. SIAM J. Control Optim, 17:365–389.
Liu, J., Chen, R., and Logvinenko, T. 2000. A theoretical framework for sequential importance sampling and resampling. Technical Report, Stanford University, Department of Statistics.
Ljung, L. 1987. System Identification—Theory for the User. Prentice Hall: Englewood Cliffs, NJ.
Mallat, S. 1989. A theory of multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:674–693.
Mumford, D. and Gidas, B. 1998. Stochastic models for generic images. Technical Report, Division of Applied Mathematics, Brown University.
Nelson, R.C. and Polana, R. 1992. Qualitative recognition of motion using temporal texture. Computer Vision, Graphics, and Image Processing. Image Understanding, 56(1):78–89.
Paget, R. and Longstaff, D. 1996. A nonparametric multiscale markov random field model for synthesising natural textures. In Fourth International Symposium on Signal Processing and its Applications, 2:744–747.
Peachey, D. 1986. Modeling waves and surf. In SIGGRAPH Conference Proceedings, pp. 65–74.
Popat, K. and Picard, R. 1993. Novel cluster-based probability model for texture synthesis, classification, and compression. In Proceedings SPIE Visual Communications and Image Processing, Boston.
Portilla, J. and Simoncelli, E. 1999. Texture representation and synthesis using correlation of complex wavelet coefficient magnitudes. In CSIC, Madrid.
Reeves, W. 1983. Particle systems: A technique for modeling a class of fuzzy objects. ACM Trans. Graphics, 2:91–108.
Rissanen, J. 1978. Modeling by shortest data description. Automatica, 14:465–471.
Saisan, P., Doretto, G., Wu, Y.N., and Soatto, S. 2001. Dynamic texture recognition. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HA.
Schödl, A., Szeliski, R., Salesin, D., and Essa, I. 2000.Video textures. In Proceedings of ACM SIGGRAPH Conference, New Orleans, LA.
Simoncelli, E., Freeman, W., Adelson, E., and Heeger, D. 1992.Shiftable multi-scale transforms. IEEE Trans. Information Theory, 38(2):587–607.
Sims, K. 1990. Particle animation and rendering using data parallel computation. Computer Graphics, 24(4):405–413.
Stam, J. and Fiume, E. 1995. Depicting fire and other gaseous phenomena using diffusion processes. InSIGGRAPHConference Proceedings, pp. 129–136.
Szummer, M. and Picard, R.W. 1996. Temporal texture modeling. In IEEE International Conference on Image Processing, Lausanne, Switzerland, vol. 3.
Van Overschee, P. and De Moor, B. 1993. Subspace algorithms for the stochastic identification problem. Automatica, 29:649– 660.
Van Overschee, P. and De Moor, B. 1994. N4sid: Subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica, 30:75–93.
Wei, L.Y. and Levoy, M. 2000. Fast texture synthesis using tree structured vector quantization. In SIGGRAPH Conference Proceedings.
Zhu, S., Wu, Y., and Mumford, D. 1997. Minimax entropy principle and its application to texture modeling. Neural Computation, 9:1627–1660.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Doretto, G., Chiuso, A., Wu, Y.N. et al. Dynamic Textures. International Journal of Computer Vision 51, 91–109 (2003). https://doi.org/10.1023/A:1021669406132
Issue Date:
DOI: https://doi.org/10.1023/A:1021669406132