Skip to main content
SpringerLink
Log in

Semantic propagation network with robust spatial context descriptors for multi-class object labeling

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a semantic propagation network model that incorporates semantic co-occurrence, absolute position context, relative position context, and appearance into a unified framework. Multi-class object labeling is modeled as a sequential process, which is formulated with the particle filter. We modify the particle filter and extend it to static data in image. A salient object is first labeled using the appearance and absolute position context, which serves as the initialization of particle filter. The semantic information of pre-labeled objects is propagated to the subsequent objects by semantic association, which is modeled by the particle sampling. Then, the appearance and relative position context are combined to infer the next object’s category. This sequential process iterates until all the objects are labeled. To represent spatial context accurately and robustly, we propose two novel spatial context descriptors. They capture the globally spatial distribution of object regions and are robust to the changes of object shapes, structures, and scales. Experimental results demonstrate the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. For simplicity, we still use \(1,2,\ldots m\) to represent \({{i}_{1}},{{i}_{2}},\ldots,{{i}_{m}}\) in the later sections.

References

  1. Agrawal R, Imielinski T, Swami A (1993) Mining association rules between sets of items in large databases. In: Proceedings of ACM SIGMOD international conference on management of data, pp 207–216

  2. Amari SI (1988) Neural theory of association and concept-formation. In: Anderson (ed) Neurocomputing: foundations of research. MIT Press, Cambridge, pp 269–281

    Google Scholar 

  3. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intel 24(4):509–522

    Article  Google Scholar 

  4. Blaschko MB, Lampert CH (2009) Object localization with global and local context kernels. Paper presented at British machine vision conference 2009

  5. Bosch A, Zisserman A, Muñoz, X (2006) Scene classification via plsa. In: Proceedings of the 9th European conference on computer vision, pp 517–530

  6. Bosch A, Zisserman A, Muñoz, X (2007) Image classification using random forests and ferns. In: Proceedings of the 11th international conference on computer vision, pp 1–8

  7. Chen X, Zhao D, Zhao Y, Lin L (2009) Accurate semantic image labeling by fast geodesic propagation. In: Proceedings of the 16th international conference on image processing, pp 4021–4024

  8. Choi MJ, Lim JJ, Torralba A, Willsky AS (2010) Exploiting hierarchical context on a large database of object categories. In: Proceedings of the 23rd computer vision and pattern recognition, pp 129–136

  9. Criminisi A (2004) Microsoft research cambridge object recognition image database. http://research.microsoft.com/en-us/projects/objectclassrecognition/

  10. Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Proceedings of workshop on statistical learning in computer vision, ECCV (2004), pp. 1–22

  11. Desai C, Ramanan D, Fowlkes C (2009) Discriminative models for multi-class object layout. In: Proceedings of the 12th international conference on computer vision, pp 229–236

  12. Galleguillos C, Rabinovich A, Belongie S (2008) Object categorization using co-occurrence, location and appearance. In: Proceedings of the 21nd conference on computer vision and pattern recognition, pp 1–8

  13. Gazzaniga M, Ivry R, Mangun G (2009) Cognitive neuroscience: the biology of the mind. Norton, New York

    Google Scholar 

  14. Gould S, Rodgers J, Cohen D, Elidan G, Koller D (2008) Multi-class segmentation with relative location prior. Int J Comput Vision 80(3):300–316

    Article  Google Scholar 

  15. He X, Zemel R, Carreira-Perpin’an M (2004) Multiscale conditional random fields for image labeling. In: Proceedings of the 17th computer vision and pattern recognition, pp 695–702

  16. Heitz G, Koller D (2008) Learning spatial context: using stuff to find things. In: Proceedings of the 10th European conference on computer vision, pp 30–43

  17. Isard M, Blake A (1998) Condensation-conditional density propagation for visual tracking. Int J Comput Vision 29(1):5–28

    Article  Google Scholar 

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  19. Lu C, Latecki LJ, Adluru N, Yang X, Ling H (2009) Shape guided contour grouping with particle filters. In: Proceedings of the IEEE 12th international conference on computer vision, pp 2288–2295

  20. Maskell S, Gordon N (2001) A tutorial on particle filters for on-line nonlinear/non-gaussian bayesian tracking. IEEE Trans Signal Process 50:174–188

    Google Scholar 

  21. Parikh D, Zitnick C, Chen T (2008) From appearance to context-based recognition: dense labeling in small images. In: Proceedings of the 21st computer vision and pattern recognition, pp 1–8

  22. Rabinovich A, Vedaldi A, Galleguillos C, Wiewiora E, Belongie S (2007) Objects in context. In: Proceedings of the 11th international conference on computer vision, pp 1–8

  23. Rudin C, Letham B, Salleb-Aouissi A, Kogan E, Madigan D (2011) Sequential event prediction with association rules. In: Proceedings of the 24th annual conference on learning theory, pp 615–634

  24. Russell B, Torralba A, Liu C, Fergus R, Freeman W (2007) Object recognition by scene alignment. In: Proceedings of the 21st annual conference on neural information processing systems (NIPS 2007)

  25. Shotton J, Winn J, Rother C, Criminisi A (2009) TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vision 81(1):2–23

    Google Scholar 

  26. Torralba A (2003) Contextual priming for object detection. Int J Comput Vision 53(2):169–191

    Article  Google Scholar 

  27. Torralba A, Murphy K, Freeman W (2004) Contextual models for object detection using boosted random fields. In: Proceedings of the 18th annual conference on neural information processing systems (NIPS 2004)

  28. Tu Z, Bai X (2010) Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans Pattern Anal Mach Intell 32(10):1744–1757

    Article  Google Scholar 

  29. Verbeek J, Triggs W (2007) Scene segmentation with crfs learned from partially labeled images. In: Proceedings of the 21st annual conference on neural information processing systems (NIPS 2007)

  30. Wang X, Liu X, Shi Z, Sui H (2011) A feature binding computational model for multi-class object categorization and recognition. Neural Comput Appl. doi:10.1007/s00521-011-0562-1

  31. Wolf L, Bileschi S (2006) A critical view of context. Int J Comput Vision 69(2):251–261

    Article  Google Scholar 

Download references

Acknowledgments

This research was funded by NSF of China grant 90920008, 91120009, and National Program on Key Basic Research Project (973 Program) grant 2012CB-316400, 2012CB-316402.

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China

    Ping Wei, Yuehu Liu, Nanning Zheng & Shaozhuo Zhai

Authors
  1. Ping Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuehu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaozhuo Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ping Wei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wei, P., Liu, Y., Zheng, N. et al. Semantic propagation network with robust spatial context descriptors for multi-class object labeling. Neural Comput & Applic 24, 1003–1018 (2014). https://doi.org/10.1007/s00521-012-1308-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1308-4

Keywords

Search

Navigation