Abstract
Dynamic texture (DT) is the annexation of an altering series of images where temporal regularity is present. Characterization, classification and identification of DTs have been investigated by many researchers in the past 2 decades. Especially, several image texture descriptors based on local binary pattern (LBP) have been enhanced for DT classification in the spatiotemporal domain. Local tetra pattern (LTrP) is an extension of LBP in which the calculation of the feature code depends on the referenced central pixel as well as its neighbor’s directions. In this work, we have proposed an LTrP-based novel spatiotemporal descriptor for characterization and subsequent identification of textures in videos. We term this descriptor as local tetra patterns on three orthogonal planes (LTrP-TOP). In our proposed work, the video frames are initially divided into three orthogonal planes on the basis of the XY, YT and XT directions. The direction of the center and neighborhood pixels are then detected in vertical and horizontal orientations using first-order derivatives. This sequence of operations computes the LTrP for each center pixel. Subsequently, we create the histogram for a particular plane by considering the LTrP codes for all the pixels in that plane. Finally, the histograms from the three orthogonal planes are concatenated for forming the final texture descriptor of the video. Comprehensive experimental assessments on benchmark DT databases (Dyntex++ and UCLA) demonstrate that the LTrP-TOP descriptor exhibits better performance than most of the other state-of-the-art DT descriptors (in terms of classification accuracy %).
Similar content being viewed by others
References
Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006). https://doi.org/10.1109/TPAMI.2006.244
Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimed. 16(8), 2099–2109 (2014). https://doi.org/10.1109/TMM.2014.2362855
Baktashmotlagh, M., Harandi, M., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2353–2366 (2014). https://doi.org/10.1109/TPAMI.2014.2339851
Chan, A.B., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 846–851 (2005). https://doi.org/10.1109/CVPR.2005.279
Chan, A.B., Vasconcelos, N.: Layered dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1862–1879 (2009). https://doi.org/10.1109/TPAMI.2009.110
Chan, A.B., Vasconcelos, N.: Variational layered dynamic textures. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1062–1069 (2009). https://doi.org/10.1109/CVPR.2009.5206556
Chetverikov, D., Péteri, R.: A brief survey of dynamic texture description and recognition. In: Computer Recognition Systems, pp. 17–26. Springer (2005)
Danelakis, A., Theoharis, T., Pratikakis, I.: A robust spatio-temporal scheme for dynamic 3d facial expression retrieval. Vis. Comput. 32(2), 257–269 (2016). https://doi.org/10.1007/s00371-015-1142-7
Derpanis, K.G.P., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1193–1205 (2012). https://doi.org/10.1109/TPAMI.2011.221
Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003). https://doi.org/10.1023/A:1021669406132
Fazekas, S., Chetverikov, D.: Analysis and performance evaluation of optical flow features for dynamic texture recognition. Image Commun. 22(7–8), 680–691 (2007). https://doi.org/10.1016/j.image.2007.05.013
Fazekas, S., Chetverikov, D.: Dynamic texture recognition using optical flow features and temporal periodicity. In: 2007 International Workshop on Content-Based Multimedia Indexing, pp. 25–32 (2007). https://doi.org/10.1109/CBMI.2007.385388
Ghanem, B., Ahuja, N.: Maximum margin distance learning for dynamic texture recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer Vision—ECCV 2010, pp. 223–236. Springer, Berlin Heidelberg (2010)
Gonçalves, W.N., Machado, B.B., Bruno, O.M.: Spatiotemporal gabor filters: a new method for dynamic texture recognition. CoRR (2012) arXiv:1201.3612
Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657–1663 (2010). https://doi.org/10.1109/TIP.2010.2044957
Murala, S., Maheshwari, R.P., Balasubramanian, R.: Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans. Image Process. 21(5), 2874–2886 (2012). https://doi.org/10.1109/TIP.2012.2188809
Nelson, R.C., Polana, R.: Qualitative recognition of motion using temporal texture. CVGIP Image Underst. 56(1), 78–89 (1992). https://doi.org/10.1016/1049-9660(92)90087-J
Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996). https://doi.org/10.1016/0031-3203(95)00067-4
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002). https://doi.org/10.1109/TPAMI.2002.1017623
Péteri, R., Chetverikov, D.: Dynamic texture recognition using normal flow and texture regularity. In: Proceedings of the Second Iberian Conference on Pattern Recognition and Image Analysis—Volume Part II, IbPRIA’05, pp. 223–230. Springer, Berlin, Heidelberg (2005). https://doi.org/10.1007/11492542_28
Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer Vision Using Local Binary Patterns, pp. E1–E2. Springer, London (2011). https://doi.org/10.1007/978-0-85729-748-8_14
Polana, R., Nelson, R.: Temporal Texture and Activity Recognition, pp. 87–124. Springer, Dordrecht (1997). https://doi.org/10.1007/978-94-015-8935-2_5
Qiao, Y., Weng, L.: Hidden markov model based dynamic texture classification. IEEE Signal Process. Lett. 22(4), 509–512 (2015). https://doi.org/10.1109/LSP.2014.2362613
Qiao, Y.L., Song, C.Y., Wang, F.S.: Wavelet-based dynamic texture classification using complex generalized gaussian distribution. In: AISS, vol. 4 (2012)
Qiao, Y.L., Song, C.Y., Wang, F.S.: Wavelet-based dynamic texture classification using gumbel distribution. Math. Probl. Eng. (2013). https://doi.org/10.1155/2013/762472
Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1651–1657 (2009). https://doi.org/10.1109/CVPR.2009.5206847
Ravichandran, A., Chaudhry, R., Vidal, R.: Categorizing dynamic textures using a bag of dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 342–353 (2013). https://doi.org/10.1109/TPAMI.2012.83
Ren, J., Jiang, X., Yuan, J.: Dynamic texture recognition using enhanced lbp features. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2400–2404 (2013). https://doi.org/10.1109/ICASSP.2013.6638085
Ren, J., Jiang, X., Yuan, J.: Noise-resistant local binary pattern with an embedded error-correction mechanism. IEEE Trans. Image Process. 22(10), 4049–4060 (2013). https://doi.org/10.1109/TIP.2013.2268976
Ren, J., Jiang, X., Yuan, J., Wang, G.: Optimizing lbp structure for visual recognition using binary quadratic programming. IEEE Signal Process. Lett. 21(11), 1346–1350 (2014). https://doi.org/10.1109/LSP.2014.2336252
Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 2, pp. II–58–II–63 (2001). https://doi.org/10.1109/CVPR.2001.990925
Shrivastava, N., Tyagi, V.: An effective scheme for image texture classification based on binary local structure pattern. Vis. Comput. 30(11), 1223–1232 (2014). https://doi.org/10.1007/s00371-013-0887-0
Smith, J.R., Lin, C.Y., Naphade, M.: Video texture indexing using spatio-temporal wavelets. In: Proceedings. International Conference on Image Processing, vol. 2, pp. II–437–II–440 (2002). https://doi.org/10.1109/ICIP.2002.1039981
Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010). https://doi.org/10.1109/TIP.2010.2042645
Tiwari, D., Tyagi, V.: Dynamic texture recognition: a review. In: Satapathy, S.C., Mandal, J.K., Udgata, S.K., Bhateja, V. (eds.) Information Systems Design and Intelligent Applications, pp. 365–373. Springer India, New Delhi (2016)
Tiwari, D., Tyagi, V.: Dynamic texture recognition based on completed volume local binary pattern. Multidimens. Syst. Signal Process. 27(2), 563–575 (2016b). https://doi.org/10.1007/s11045-015-0319-6
Tiwari, D., Tyagi, V.: A novel scheme based on local binary pattern for dynamic texture recognition. Comput. Vis. Image Underst. 150, 58–65 (2016). https://doi.org/10.1016/j.cviu.2016.04.010
Tiwari, D., Tyagi, V.: Dynamic texture recognition using multiresolution edge-weighted local structure pattern. Comput. Electr. Eng. 62, 485–498 (2017a). https://doi.org/10.1016/j.compeleceng.2016.11.008
Tiwari, D., Tyagi, V.: Improved weber’s law based local binary pattern for dynamic texture recognition. Multimed. Tools Appl 76(5), 6623–6640 (2017b). https://doi.org/10.1007/s11042-016-3362-x
Tung, T., Matsuyama, T.: Invariant shape descriptor for 3d video encoding. Vis. Comput. 31(3), 311–324 (2015). https://doi.org/10.1007/s00371-014-0925-6
Wang, Y., Hu, S.: Exploiting high level feature for dynamic textures recognition. Neurocomputing 154(C), 217–224 (2015). https://doi.org/10.1016/j.neucom.2014.12.001
Wang, Y., Hu, S.: Chaotic features for dynamic textures recognition. Soft Comput. 20(5), 1977–1989 (2016). https://doi.org/10.1007/s00500-015-1618-4
Xu, Y., Quan, Y., Ling, H., Ji, H.: Dynamic texture classification using dynamic fractal analysis. In: 2011 International Conference on Computer Vision, pp. 1219–1226 (2011). https://doi.org/10.1109/ICCV.2011.6126372
Xu, Y., Huang, S., Ji, H., FermüLler, C.: Scale-space texture description on sift-like textons. Comput. Vis. Image Underst. 116(9), 999–1013 (2012). https://doi.org/10.1016/j.cviu.2012.05.003
Zhang, B., Gao, Y., Zhao, S., Liu, J.: Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans. Image Process. 19(2), 533–544 (2010). https://doi.org/10.1109/TIP.2009.2035882
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007). https://doi.org/10.1109/TPAMI.2007.1110
Zhao, G., Pietikäinen, M.: Dynamic texture recognition using volume local binary patterns. In: Vidal, R., Heyden, A., Ma, Y. (eds.) Dynamical Vision, pp. 165–177. Springer, Berlin, Heidelberg (2007)
Zhao, G., Ahonen, T., Matas, J., Pietikainen, M.: Rotation-invariant image and video description with local binary pattern features. IEEE Trans. Image Process. 21(4), 1465–1477 (2012). https://doi.org/10.1109/TIP.2011.2175739
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Amit, Raman, B. & Sadhya, D. Dynamic texture recognition using local tetra pattern—three orthogonal planes (LTrP-TOP). Vis Comput 36, 579–592 (2020). https://doi.org/10.1007/s00371-019-01643-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-019-01643-4