Abstract
In this paper, we are tackling the problem of distinguishing scenes, including static and dynamic scenes. We propose a framework of scene recognition, based on bag of visual words and topic model. We achieve the task using the topic model by belief propagation (TMBP), which belongs to the family of the latent Dirichlet allocation model. We also extend the TMBP model, called as the knowledge TMBP model, by introducing the prior information of visual words and scenes. Experimental results on the static and dynamic scenes demonstrated that our proposed framework is effective and efficient. The scene semantics can be obtained from two levels of visual words and topics in our framework. Our result significantly outperforms the others using low-level visual features, such as spatial, temporal and spatiotemporal features.










Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alqasrawi Y, Neagu D, Cowling P (2009) Natural scene image recognition by fusing weighted colour moments with bag of visual patches on spatial pyramid layout. Proceedings of the 9th international conference on intelligent systems design and applications, ISDA, IEEE Computer Society, Pisa, Italy, Nov 30–Dec 2, 2009, pp 140– 145
Battiato S, Farinella G, Gallo G, Ravi D (2010) Exploiting textons distributions on spatial hierarchy for scene classification. EURASIP J Image Video Process, special issue on multimedia modeling, Jan 2010, pp 1–13
Bisho CM (2006) Pattern Recognition and Machine Learning. Springer
Blei D, Ng A, Jordan M (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022
Bosch A, Munoz X, Marti R (2007) Which is the best way to organize/classify images by content? Image Vis Comput 5(6):778–791
Bosch A, Zisserman A, Munoz X (2008) Scene classification using a hybrid generative/discriminative approach. IEEE Trans Pattern Anal Mach Intell (PAMI) 30(4):712–727
Bosch A, Zisserman A, Munoz X (2007) Representing shape with a spatial pyramid kernel. Proceedings of the 6th ACM international conference on image and video retrieval, CIVR, Amsterdam, The Netherlands, July 9–11, 2007, pp 401–408
Cao Y, Wang C, Li Z, Zhang L, Zhang L (2010) Spatial bag-of-features. In: CVPR, June 13–18, 2010, San Francisco, CA, pp 3352–3359
Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision (ECCV). Prague Czech Republic, pp 1–22
Derpanis KG, Lecce M, Daniilidis K, Wildes RP (2012) Dynamic scene understanding: the role of orientation features in space and time in scene classification. In: CVPR, Providence, RI, USA, June 16–21 2012, pp 1306–1313
Feichtenhofer C, Pinz A, Wildes RP (2013) Spacetime forests with complementary features for dynamic scene recognition. In: Proceedings of the British machine vision conference (BMVC)
Fei-Fei L, Fergus R (2003) Bayesian approach to unsupervised one-shot learning of object categories. In: ICCV, Nice, France, Oct 13–16 2003, vol 2, pp 1134–1141
Fei-Fei L, Perona P (2005) A bayesian hierarchical model for learning natural scene categories. In: CVPR, San Diego, CA, USA, June 20–26 2005, vol 2, pp 524–531
Grossberg S, Huang T (2009) ARTSCENE: a neural system for natural scene classification. J Vis 9(4):1–19
Harada T, Ushiku Y, Yamashita Y, Kuniyoshi Y (2011) Discriminative spatial pyramid. In: CVPR, Providence, RI, USA, June 20–25 2011, pp 1617–1624
Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1–2):177–196
Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469
Jiang YG, Ngo CW, Yang J (2007) Towards optimal bag-of-features for object categorization and semantic video retrieval. Proceedings of the 6th ACM international conference on image and video retrieval, CIVR, Amsterdam, The Netherlands, July 9–11, 2007, pp 494– 501
Julien SL, Sha F, Jordan MI (2008) DiscLDA: discriminative learning for dimensionality reduction and classification. In: NIPS, pp 897– 904
Khan F, van de Weijer J, Vanrell M (2009) Top-down color attention for object recognition. In: ICCV, Kyoto, Japan, Sept 27–Oct 4, 2009, pp 979–986
Kuettel D, Breitenstein M, Gool LV, Ferrari V (2010) What’s going on? Discovering spatio-temporal dependencies in dynamic scenes, In: CVPR, San Francisco, CA, USA, June 13–18 2010, pp 1951–1958
Lampert CH, Blaschko MB, Hofmann T (2008) Beyond sliding windows: object localization by efficient subwindow search. In: CVPR, Anchorage, Alaska, USA, June 24–26, 2008, pp 1–8
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognising natural scene categories. In: CVPR, New York, USA, June 17–22, 2006, pp 2169–2178
Li H, Wang F, Zhang S (2011) Global and local features based topic model for scene recognition. 2011 IEEE nternational conference on systems, man, and cybernetics (SMC), 9–12 Oct 2011, Anchorage, AK, pp 532–537
Marszalek M, Laptev I, Schmid C (2009) Actions in context. In: CVPR, Miami, FL, USA, June 20–25 2009, pp 2929–2936
Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. In: CVPR, New York, USA, June 17–22, 2006, pp 2161–2168
Niu Z, Hua G, Gao X, Tian Q (2011) Spatial-discLDA for visual recogniton, In: CVPR, June 20–25, 2011, Providence, RI, pp 1769–1776
Niu Z, Hua G, Gao X, Tian Q (2012) Context aware topic model for scene recognition, In: CVPR, June 16–21, 2012 Providence, RI, pp 2743–2750
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. IJCV 42(3):145–175
Perronnin F (2008) Universal and adapted vocabularies for generic visual categorization. PAMI 30(7):1243–1256
Quelhas P, Monay F, Odobez JM, Gatica-Perez D, Tuytelaars T, Van Gool L (2005) Modeling scenes with local descriptors and latent aspects, proceedings of IEEE international conference on computer vision ICCV, Beijing, China, Oct 17–21, 2005, pp 883–890
Quelhas P, Odobez J (2007) Multi-level local descriptor quantization for bag-of-visterms image representation. Proceedings of the 6th ACM international conference on image and video retrieval, Amsterdam, The Netherlands, July 9–11, 2007, pp 242–249
Ramos J (2003) Using tf-idf to determine word relevance in document queries. In: Proceedings of the first instructional conference on machine learning, Piscataway, New Jersey, USA, Dec 3–8 2003
Ravichandran A, Chaudhry R, Vidal R (2013) View-invariant dynamic texture recognition using a bag of dynamical systems. PAMI 35(2):342–353
Shroff N, Turaga P, Chellappa R (2010) Moving vistas: exploiting motion for describing scenes. In: CVPR, San Francisco, CA, USA, June 13–18 2010, pp 1911–1918
Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: ICCV, Nice, France, Oct 13–16 2003, vol 2, pp 1470–1477
Sivic J, Russell B, Efros AA, Zisserman A, Reeman B (2005) Discovering objects and their location in images. In: ICCV, Oct 17–21, 2005. Beijing, China, pp 370–377
Sudderth EB, Torralba A, Freeman WT, Willsky AS (2005) Learning hierarchical models of scenes, objects, and parts. In: ICCV, 17–21 Oct 2005, Vol 2, Beijing, China, pp 1331–1338
Theriault C, Thome N, Cord M (2013) Dynamic scene classification: learning motion descriptors with slow features analysis. In: CVPR, Portland, OR, USA, June 23–28 2013, pp 2603–2610
Wang X, Ma KT, Ng GW et al (2011) Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int J Comp Vis 95(3):287–312
Wu J, Rehg J (2011) CENTRIST: a visual descriptor for scene categorization. PAMI 33(8):1489–1501
Wu Z, Ke Q, Sun J, Shum HY (2009) A multi-sample, multi-tree approach to bag-of-words image representation for image retrieval. In: ICCV, Kyoto, Japan, Sept 27–Oct 4, 2009, pp 1992–1999
Wu J, Rehg J (2009) Beyond the Euclidean distance: creating effective visual codebooks using the histogram intersection kernel. In: ICCV, Kyoto, Japan, Sept 27–Oct 4, 2009, pp 630–637
Yang J, Jiang YG, Hauptmann AG, Ngo CW (2007) Evaluating bag-of-visual-words representations in scene classification. Proceedings of the 9th ACM international workshop on multimedia information retrieval, ACM MIR, University of Augsburg, Germany, Sept 28–29, 2007, pp 197–206
Zeng J, Cheung WK-W, Liu J (2013) Learning topic models by belief propagation. PAMI 35(5):1121–1134
Zhang Z (2008) Reasearch of object categories using bag of synonyms model, Master degree theses, Beijing Capital University, pp 9–15
Zhou H, Yuan Y, Shi C (2009) Object tracking using SIFT features and mean shift. Comp Vis Image Underst 113(3):345–352
Zhu L, Zhang A (2002) Theory of keyblock-based image retrieval. ACM Trans Inf Syst (TOIS) 20(2):224–257
Acknowledgments
This work is supported by National Natural Science Foundation of China (Grant Nos. 61272258, 61170124, 61301299, 61170020, 61272005).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by L. Xie.
Rights and permissions
About this article
Cite this article
Liu, C., Lin, H., Gong, S. et al. Learning topic of dynamic scene using belief propagation and weighted visual words approach. Soft Comput 19, 71–84 (2015). https://doi.org/10.1007/s00500-014-1384-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-014-1384-8