Abstract
Region Labeling is to automatically assign semantic labels to the corresponding image regions. Most of the previous works focus on exploiting low level visual features, particularly visual spatial contextual information, to address the problem. However, very few work explore high level semantic information of the whole image to deal with the problem. In this paper, we propose a new region labeling approach by integrating both visual spatial and semantic contextual information into a unified model. In our method, region labeling is regarded as a multi-class classification problem. For each semantic concept, we train a Conditional Random Field (CRF) model respectively. It consists of both the region grid sub-graph and the co-occurred semantic label sub-graph. In our model, the integration of the two kinds of contextual information brings reinforcement effect on the improvement of region labeling. The experiments are conducted on two commonly used benchmark datasets and the experimental results show that our method achieves the best performance compared with the strong baselines and the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Athanasiadis, T., Mylonas, P., Avrithis, Y.: A context-based region labeling approach for semantic image segmentation. In: Avrithis, Y., Kompatsiaris, Y., Staab, S., O’Connor, N.E. (eds.) SAMT 2006. LNCS, vol. 4306, pp. 212–225. Springer, Heidelberg (2006)
Corso, J.J., Yuille, A., Tu, Z.: Natural image labeling by dynamic hierarchical computing. In: CVPR (2008)
Doretto, G., Yao, Y.: Region moments: Fast invariant descriptors for detecting samll image strctures. In: CVPR (2010)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004)
Feng, S., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and vedio annotation. In: CVPR (2004)
Han, Y., Wu, F., Shao, J., Tian, Q., Zhuang, Y.: Graph-guided sparse reconstruction for region tagging. In: CVPR (2012)
Ji, C., Zhou, X., Lin, L., Yang, W.: Labeling images by integrating sparse multiple distance learning and semantic context modeling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 688–701. Springer, Heidelberg (2012)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: ICML (2001)
Lin, Z., Chan, W., He, K., Zhou, X., Wang, M.: Conditional random fileds for image region labeling with global observation. In: Huet, B., Ngo, C.-W., Tang, J., Zhou, Z.-H., Hauptmann, A.G., Yan, S. (eds.) PCM 2013. LNCS, vol. 8294, pp. 586–597. Springer, Heidelberg (2013)
Liu, X., Cheng, B., Yan, S.: Label to region by bi-layer sparsity priors. In: MM (2009)
Llorente, A., Motta, E., Rüger, S.: Exploring the semantics behind a collection to improve automated image annotation. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T. (eds.) CLEF 2009. LNCS, vol. 6242, pp. 307–314. Springer, Heidelberg (2010)
Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)
Rubinstein, M., Liu, C., Freeman, W.T.: Annnotaion propagation in large image databases via dense image correspondence. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 85–99. Springer, Heidelberg (2012)
Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)
Sutton, C., McCallum, A.: An introduction to conditional random fields. Foundations and Trends in Machine Learning 4, 267–373 (2011)
Tang, J., Hong, R., Yan, S., Chua, T., Qi, G., Jain, R.: Image annotation by knn-sparse graph-based label propagation over noisily-tagged web images. ACM Transactions on Intelligent Systems and Techology 2 (2011)
Toyota, T., Hasegawa, O.: Random field model for integration of local information and global information. IEEE Tranaction on PAMI 30(8), 1483–1489 (2008)
Wang, H., Huang, H., Ding, C.: Image annotation using bi-relational graph of images and semantic labels. In: CVPR (2011)
Xiang, Y., Zhou, X., Chua, T., Ngo, C.: A revisit of generative model for automatic image annotation using markov random fields. In: CVPR (2009)
Xiang, Y., Zhou, X., Chua, T., Ngo, C.: Semantic context modeling with maximal margin conditional random fields for automatic image annotation. In: CVPR (2010)
Zheng, J., Jiang, Z.: Tag taxonomy aware dictionary learning for region tagging. In: CVPR (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
He, K., Chan, W., Zhu, G., Lin, L., Zhou, X. (2014). Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts. In: Ooi, W.T., Snoek, C.G.M., Tan, H.K., Ho, CK., Huet, B., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2014. PCM 2014. Lecture Notes in Computer Science, vol 8879. Springer, Cham. https://doi.org/10.1007/978-3-319-13168-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-13168-9_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13167-2
Online ISBN: 978-3-319-13168-9
eBook Packages: Computer ScienceComputer Science (R0)