skip to main content
10.1145/2396761.2398428acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Modeling semantic relations between visual attributes and object categories via dirichlet forest prior

Published: 29 October 2012 Publication History

Abstract

In this paper, we deal with two research issues: the automation of visual attribute identification and semantic relation learning between visual attributes and object categories. The contribution is two-fold, firstly, we provide uniform framework to reliably extract both categorical attributes and depictive attributes. Secondly, we incorporate the obtained semantic associations between visual attributes and object categories into a text-based topic model and extract descriptive latent topics from external textual knowledge sources. Specifically, we show that in mining natural language descriptions from external knowledge sources, the relation between semantic visual attributes and object categories can be encoded as Must-Links and Cannot-Links, which can be represented by Dirichlet-Forest prior. To alleviate the workload of manual supervision and labeling in image categorization process, we introduce a semi-supervised training framework using soft-margin semi-supervised SVM classifier. We also show that the large-scale image categorization results can be significantly improved by combining automatically acquired visual attributes. Experimental results show that the proposed model achieves better ability in describing object-related attributes and makes the inferred latent topics more descriptive.

References

[1]
Andrzejewski D, Zhu X, Craven M. (2009) Incorporating domain knowledge into topic modeling via Dirichlet Forest priors. In Proceedings of the 26th Annual international Conference on Machine Learning (Montreal, Quebec, Canada, June 14 - 18, 2009). ICML '09, vol. 382. ACM, New York, NY, 25--32.
[2]
B. Schoelkopf, A. Smola, Learning with Kernels, 644, MIT Press, Cambridge, MA (2002).
[3]
C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.
[4]
C. Siagian and L. Itti, Rapid Biologically-Inspired Scene Classification Using Features Shared with Visual Attention, IEEE TPAMI, pp. 300--312, 2007.
[5]
C. Wang, L. Zhang, and H. Zhang. Learning to reduce the semantic gap in web image retrieval and annotation. In SIGIR, 2008.
[6]
D. G. David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 2004.
[7]
H. Wang, X. Jiang, L.-T. Chia, and A.-H. Tan. Ontology enhanced web image retrieval: aided by wikipedia & spreading activation theory. In MIR, 2008. 2
[8]
J. Deng, A. Berg, K. Li and L. Fei-Fei, What does classifying more than 10,000 image categories tell us? Proceedings of the 12th European Conference of Computer Vision (ECCV). 2010.
[9]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, ImageNet: A Large-Scale Hierarchical Image Database. IEEE Computer Vision and Pattern Recognition (CVPR), 2009.
[10]
J. Platt. Probabilistic outputs for support vector machines and comparison to regularize likelihood methods. In Advances in Large Margin Classifiers, pages 61--74, 2000.
[11]
L. Li, C. Wang, Y. Lim, D. Blei and L. Fei-Fei. Building and Using a Semantivisual Image Hierarchy. IEEE Computer Vision and Pattern Recognition (CVPR). 2010.
[12]
M. Belkin, P. Niyogi & V. Sindhwani (2004) Manifold Regularization: A Geometric Framework for Learning for Examples. Technical Report TR-2004- 06, Dept. of Computer Science, Univ. of Chicago.
[13]
M. Rohrbach, M. Stark, G. Szarvas, I. Gurevych, and B. Schiele, What helps where - and why? Semantic relatedness for knowledge transfer. In Proceedings of CVPR. 2010, 910--917.
[14]
O. Russakovsky and L. Fei-Fei, Attribute Learning in Large-scale Datasets. Proceedings of the 12th European Conference of Computer Vision (ECCV), 1st International Workshop on Parts and Attributes. 2010.
[15]
T. Deselaers and V. Ferrari. Visual and Semantic Similarity in ImageNet. IEEE Computer Vision and Pattern Recognition (CVPR 2011), Colorado Springs, CO, USA, pages 1777--1784 June 2011.
[16]
T. P. Minka. Estimating a dirichlet distribution. http://research. Microsoft.com/en-us/um/people/minka/papers/dirichlet, 2009.
[17]
T. P. Minka, "The dirichlet-tree distribution," in http://research.microsoft.com/~minka/papers/dirichlet/minkadirtree.pdf, 1999.
[18]
X. Chen, X. Hu, Z. Zhou, C. Lu, G. Rosen, T. He, E.K. Park, A Probabilistic Topic-Connection Model for Automatic Image Annotation, the 19th ACM Conference on Information and Knowledge Management (ACM CIKM 2010), Oct 26--29, 2010, Toronto, Canada. pp. 899--908
[19]
X. Hu, X. Zhang, C. Lu, E.K. Park, and X. Zhou, Exploiting Wikipedia as external knowledge for document clustering. In Proceedings of KDD. 2009, 389--396
[20]
X. Wang, T. X. Han and S. Yan, An HOG-LBP Human Detector with Partial Occlusion Handling, IEEE International Conference on Computer Vision (ICCV 2009), Kyoto, 2009.
[21]
Y. Cheng: Mean Shift, Mode Seeking, and Clustering. IEEE Transaction on Pattern Analysis and Machine Intelligence 17(8) (1995) 790--799

Cited By

View all
  • (2024)A Guided Gaussian-Dirichlet Random Field for Scientist-in-the-Loop Inference in Underwater Robotics2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10611290(9448-9454)Online publication date: 13-May-2024
  • (2017)Guided HTMIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.262579029:2(330-343)Online publication date: 1-Feb-2017

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management
October 2012
2840 pages
ISBN:9781450311564
DOI:10.1145/2396761
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. dirichlet-forest prior
  2. topic model
  3. visual attribute identification

Qualifiers

  • Research-article

Conference

CIKM'12
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Guided Gaussian-Dirichlet Random Field for Scientist-in-the-Loop Inference in Underwater Robotics2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10611290(9448-9454)Online publication date: 13-May-2024
  • (2017)Guided HTMIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.262579029:2(330-343)Online publication date: 1-Feb-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media