Skip to main content
SpringerLink
Log in

Bayesian network based semantic image classification with attributed relational graph

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Semantic image classification is a hot issue of image mining. Information of spatial relations between objects in an image is one of the important semantic information of an image. However, the previous researches have not made full use of the spatial relations for image modeling and classification. In addition, to classify the images with Bayesian network, the accuracy of conditional probability estimation may be insufficient, because the learning methods of spatial contextual models have usually used a limited number of training samples. In this work, the semantic image modeling based on attributed relational graph has been proposed, in which the distance measure method between images was presented, therefore the object information and spatial relational information could be fully utilized. Then, the semantic distance between images based on attributed relational graph could be calculated for the support vector machine to obtain the joint conditional probability distribution of Bayesian network. Therefore the probabilistic estimation problem under the sparse training samples could be solved, and the accuracy of semantic image classification with Bayesian network was improved. Experimental results show the validity and reliability of this proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Aksoy S, Koperski K, Tusk C et al (2005) Learning Bayesian classifiers for scene classification with a visual grammar [J]. IEEE Trans Geosci Remote Sens 43(3):581–589. doi:10.1109/TGRS.2004.839547

    Article  Google Scholar 

  2. Aksoy S, Tusk C, Koperski K, et al. (2003) Scene modeling and image mining with a visual grammar [J]. Frontiers of Remote Sensing Information Processing, 35–62. doi:10.1142/9789812796752_0003

  3. Athanasiadis T, Mylonas P, Avrithis Y et al (2007) Semantic image segmentation and object labeling [J]. Circ Syst Video Technol IEEE Trans 17(3):298–312. doi:10.1109/TCSVT.2007.890636

    Article  Google Scholar 

  4. Berretti S, Del Bimbo A, Vicario E (2001) Efficient matching and indexing of graph models in content-based retrieval [J]. IEEE Trans Patt Anal Mach Intell 23(10):1089–1105. doi:10.1109/34.954600

    Article  Google Scholar 

  5. Bosch A, Zisserman A, Muoz X (2008) Scene classification using a hybrid generative/discriminative approach [J]. Patt Anal Mach Intell IEEE Trans 30(4):712–727. doi:10.1109/TPAMI.2007.70716

    Article  Google Scholar 

  6. Boutell MR, Luo J, Brown CM (2007) Scene parsing using region-based generative models [J]. IEEE Trans Multimedia 9(1):136–146. doi:10.1109/tmm.2006.886372

    Article  Google Scholar 

  7. Bruzzone L, Persello C (2009) A novel context-sensitive semisupervised SVM classifier robust to mislabeled training samples [J]. IEEE Trans Geosci Remote Sens 47(7):2142–2154. doi:10.1109/tgrs.2008.2011983

    Article  Google Scholar 

  8. Caputo B, Jie L (2009) A performance evaluation of exact and approximate match kernels for object recognition [J]. Electron Lett Comp Vision Image Anal 8(3):15–26

    Google Scholar 

  9. Carbonetto P, de Freitas N, Barnard K (2004) A statistical model for general contextual object recognition[C]. Computer Vision - ECCV 2004. Lecture Notes in Computer Science Volume 3021: 350–362. doi:10.1007/978-3-540-24670-1_27

  10. Chang E, Goh K, Sychay G et al (2003) CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines [J]. Circ Syst Video Technol IEEE Trans 13(1):26–38. doi:10.1109/TCSVT.2002.808079

    Article  Google Scholar 

  11. Cheng H, Wang R (2010) Semantic modeling of natural scenes based on contextual Bayesian networks [J]. Patt Recog 43(12):4042–4054. doi:10.1016/j.patcog.2010.06.004

    Article  MATH  Google Scholar 

  12. Choi MJ, Torralba A, Willsky AS (2012) A tree-based context model for object recognition [J]. Patt Anal Mach Intell IEEE Trans 34(2):240–252. doi:10.1109/TPAMI.2011.119

    Article  Google Scholar 

  13. Duchenne O, Joulin A, Ponce J (2011) A graph-matching kernel for object categorization[C]//Proc IEEE Int Conf Comp Vis (ICCV), 1792–1799. doi:10.1109/ICCV.2011.6126445

  14. Fei-Fei L, Perona P (2005) A Bayesian hierarchical model for learning natural scene categories [C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2: 524–531. doi:10.1109/CVPR.2005.16

  15. Galleguillos C, Belongie S (2010) Context based object categorization: a critical survey [J]. Comp Vision Image Underst 114(6):712–722. doi:10.1016/j.cviu.2010.02.004

    Article  Google Scholar 

  16. Galleguillos C, Rabinovich A, Belongie S (2008) Object categorization using co-occurrence, location and appearance [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8. doi:10.1109/CVPR.2008.4587799

  17. Harchaoui Z, Bach F (2007) Image classification with segmentation graph kernels[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), 1–8. doi:10.1109/CVPR.2007.383049

  18. Huang Y, Huang K, Wang C, et al. (2011) Exploring relations of visual codes for image classification[C]//Computer Vision and Pattern Recognition (CVPR). IEEE Conference on. IEEE, 1649–1656. doi:10.1109/CVPR.2011.5995655

  19. Jin B, Hu W, Wang H (2012) Image classification based on pLSA fusing spatial relationships between Topics [J]. Signal Process Lett IEEE 19(3):151–154. doi:10.1109/LSP.2012.2184091

    Article  MathSciNet  Google Scholar 

  20. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories[C]//Computer Vision and Pattern Recognition. IEEE Comp Soc Conf IEEE 2:2169–2178. doi:10.1109/CVPR.2006.68

    Google Scholar 

  21. Liu S, Xu D, Feng S (2011) Region contextual visual words for scene categorization [J]. Expert Syst Appl 38(9):11591–11597. doi:10.1016/j.eswa.2011.03.037

    Article  Google Scholar 

  22. Luo J, Savakis AE, Singhal A (2005) A Bayesian network-based framework for semantic image understanding [J]. Patt Recog 38(6):919–934. doi:10.1016/j.patcog.2004.11.001

    Article  Google Scholar 

  23. Papadopoulos GT, Saathoff C, Escalante HJ et al (2011) A comparative study of object-level spatial context techniques for semantic image analysis [J]. Comp Vision Image Underst 115(9):1288–1307. doi:10.1016/j.cviu.2011.05.005

    Article  Google Scholar 

  24. Park BG, Lee KM, Lee SU et al (2003) Recognition of partially occluded objects using probabilistic ARG (attributed relational graph)-based matching [J]. Comp Vision Image Underst 90(3):217–241. doi:10.1016/S1077-3142(03)00049-3

    Article  MATH  Google Scholar 

  25. Qi GJ, Hua XS, Rui Y et al (2010) Image classification with kernelized spatial-context [J]. IEEE Trans Multimedia 12(4):278–287. doi:10.1109/TMM.2010.2046270

    Article  Google Scholar 

  26. Rabinovich A, Vedaldi A, Galleguillos C, et al. (2007) Objects in context[C]//In Proc. of 11th International Conference on Computer Vision, 1–8. doi:10.1109/ICCV.2007.4408986

  27. Serrano N, Savakis AE, Luo J (2004) Improved scene classification using efficient low-level features and semantic cues [J]. Patt Recog 37(9):1773–1784. doi:10.1016/j.patcog.2004.03.003

    Article  MATH  Google Scholar 

  28. Shen J, Shepherd J, Ngu AHH (2005) Semantic-sensitive classification for large image libraries[C]//Multimedia Modelling Conference, 2005. MMM 2005. Proceedings of the 11th International. IEEE, 340–345. doi:10.1109/MMMC.2005.66

  29. Singhal A, Luo J, Zhu W. Probabilistic spatial context models for scene content understanding [C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, 1: 235–241. doi:10.1109/CVPR.2003.1211359

  30. Su Y, Jurie F (2012) Improving image classification using semantic attributes [J]. Int J Comp Vision 100(1):59–77. doi:10.1007/s11263-012-0529-4

    Article  Google Scholar 

  31. Vailaya A, Figueiredo MAT, Jain AK et al (2001) Image classification for content-based indexing [J]. Image Proc IEEE Trans 10(1):117–130. doi:10.1109/83.892448

    Article  MATH  Google Scholar 

  32. Vogel J, Schiele B (2007) Semantic modeling of natural scenes for content-based image retrieval [J]. Int J Comp Vision 72(2):133–157. doi:10.1007/s11263-006-8614-1

    Article  Google Scholar 

  33. Yang J, Yu K, Gong Y, et al. (2009) Linear spatial pyramid matching using sparse coding for image classification [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 1794–1801. doi:10.1109/CVPR.2009.5206757

  34. Yang L, Zheng N, Yang J (2011) A unified context assessing model for object categorization [J]. Comp Vision Image Underst 115(3):310–322. doi:10.1016/j.cviu.2010.10.011

    Article  Google Scholar 

Download references

Acknowledgments

This work was partly supported by the 973 Program (2013CB329504), NSF of China (No. 61272261), NSF of Zhejiang (Y1110152), and STD of Zhejiang (2012C21002).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Chang-Yong Ri & Min Yao

  2. Institute of Information Science, Kim Il Song University, Pyongyang, 190016, Democratic People’s Republic of Korea

    Chang-Yong Ri

Authors
  1. Chang-Yong Ri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chang-Yong Ri.

Appendix: Derivation of image classification formula with Bayesian network

Appendix: Derivation of image classification formula with Bayesian network

$$ \begin{array}{c}\hfill {C}^{\ast }= arg\underset{C_n}{ \max }p\left({C}_n\Big|A,W\right)\hfill \\ {}\hfill = arg\underset{C_n}{ \max}\frac{p\left({C}_n\right)p\left(A,W\Big|{C}_n\right)}{p\left(A,W\right)}= arg\underset{C_n}{ \max}\frac{p\left({C}_n\right)p\left(A\Big|{C}_n\right)p\left(W\Big|{C}_n,A\right)}{p\left(A,W\right)}\hfill \\ {}\hfill = arg\underset{C_n}{ \max }p\left({C}_n\right)p\left(A\Big|{C}_n\right)p\left(W\Big|{C}_n,A\right)\hfill \\ {}\hfill = arg\underset{C_n}{ \max }p\left({C}_n\right)p\left({A}_1,{A}_2,\dots, {A}_M\Big|{C}_n\right)p\left({W}_{12},{W}_{13},\dots, {W}_{M-1M}\Big|{C}_n,A\right)\hfill \\ {}\hfill = arg\underset{C_n}{ \max }p\left({C}_n\right){\displaystyle \prod_{i=1}^Mp\left({A}_i\Big|{C}_n\right)}{\displaystyle \prod_{\begin{array}{c}\hfill i,j=1\hfill \\ {}\hfill i\ne j,i<j\hfill \end{array}}^Mp\left({W}_{ij}\Big|{C}_n,{A}_i,{A}_j\right)}\hfill \\ {}\hfill = arg\underset{C_n}{ \max }p\left({C}_n\right){\displaystyle \prod_{i=1}^Mp\left({A}_i\Big|{C}_n\right)}{\displaystyle \prod_{\begin{array}{c}\hfill i,j=1\hfill \\ {}\hfill i\ne j,i<j\hfill \end{array}}^Mp\left({W}_{1, ij},{W}_{2, ij},{W}_{3, ij}\Big|{C}_n,{A}_i,{A}_j\right)}\hfill \\ {}\hfill = arg\underset{C_n}{ \max }p\left({C}_n\right){\displaystyle \prod_{i=1}^Mp\left({A}_i\Big|{C}_n\right)}{\displaystyle \prod_{\begin{array}{c}\hfill i,j=1\hfill \\ {}\hfill i\ne j,i<j\hfill \end{array}}^M{\displaystyle \prod_{k=1}^3p\left({W}_{k, ij}\Big|{C}_n,{A}_i,{A}_j\right)}}\hfill \end{array} $$

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ri, CY., Yao, M. Bayesian network based semantic image classification with attributed relational graph. Multimed Tools Appl 74, 4965–4986 (2015). https://doi.org/10.1007/s11042-014-1858-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-1858-9

Keywords

Search

Navigation