Abstract
Dynamic textures (DT) are videos of non-rigid dynamical objects, such as fire and waves, which constantly change their shape and appearance over time. Most of the prior work on DT analysis dealt with the classification of videos of a single DT or the segmentation of videos containing multiple DTs. In this paper, we consider the problem of joint segmentation and categorization of videos of multiple DTs under varying viewpoint, scale, and illumination conditions. We formulate this problem of assigning a class label to each pixel in the video as the minimization of an energy functional composed of two terms. The first term measures the cost of assigning a DT category to each pixel. For this purpose, we introduce a bag of dynamic appearance features (BoDAF) approach, in which we fit each video with a linear dynamical system (LDS) and use features extracted from the parameters of the LDS for classification. This BoDAF approach can be applied to the whole video, thus providing a framework for classifying videos of a single DT, or to image patches (superpixels), thus providing the cost of assigning a DT category to each pixel. The second term is a spatial regularization cost that encourages nearby pixels to have the same label. The minimization of this energy functional is carried out using the random walker algorithm. Experiments on existing databases of a single DT demonstrate the superiority of our BoDAF approach with respect to state-of-the art methods. To the best of our knowledge, the problem of joint segmentation and categorization of videos of multiple DTs has not been addressed before, hence there is no standard database to test our method. We therefore introduce a new database of videos annotated at the pixel level and evaluate our approach on this database with promising results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Doretto, G., Cremers, D., Favaro, P., Soatto, S.: Dynamic texture segmentation. In: IEEE Int. Conf. on Computer Vision, pp. 44–49 (2003)
Ghoreyshi, A., Vidal, R.: Segmenting dynamic textures with ising descriptors, ARX models and level sets. In: Vidal, R., Heyden, A., Ma, Y. (eds.) WDV 2005/2006. LNCS, vol. 4358, pp. 127–141. Springer, Heidelberg (2007)
Chan, A., Vasconcelos, N.: Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE Trans. on Pattern Analysis and Machine Intelligence 30, 909–926 (2008)
Chan, A., Vasconcelos, N.: Layered dynamic textures. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1862–1879 (2009)
Chan, A., Vasconcelos, N.: Variational layered dynamic textures. In: IEEE Conf. on Computer Vision and Pattern Recognition (2009)
Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. II, pp. 58–63 (2001)
Chan, A., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 846–851 (2005)
Vishwanathan, S., Smola, A., Vidal, R.: Binet-Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. Int. Journal of Computer Vision 73, 95–119 (2007)
Vidal, R., Favaro, P.: Dynamicboost: Boosting time series generated by dynamical systems. In: IEEE Int. Conf. on Computer Vision (2007)
Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: IEEE Conf. on Computer Vision and Pattern Recognition (2009)
Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. Journal of Computer Vision 51, 91–109 (2003)
Ravichandran, A., Vidal, R.: Video registration using dynamic textures. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 514–526. Springer, Heidelberg (2008)
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. Journal of Computer Vision 20, 91–110 (2003)
Dance, C., Willamowski, J., Fan, L., Bray, C., Csurka, G.: Visual categorization with bags of keypoints. In: European Conf. on Computer Vision (2004)
Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: IEEE Int. Conf. on Computer Vision, pp. 1470–1477 (2003)
Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: Simplemkl. Journal of Machine Learning Research 9, 2491–2521 (2008)
Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)
Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: IEEE Int. Conf. on Computer Vision, pp. 105–112 (2001)
Grady, L.: Multilabel random walker image segmentation using prior models. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 763–770 (2005)
Péteri, R., Huskies, M., Fazekas, S.: Dyntex: A comprehensive database of dynamic textures (Online Dynamic Texture Database)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ravichandran, A., Favaro, P., Vidal, R. (2011). A Unified Approach to Segmentation and Categorization of Dynamic Textures. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6492. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19315-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-19315-6_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19314-9
Online ISBN: 978-3-642-19315-6
eBook Packages: Computer ScienceComputer Science (R0)