Hanwang Zhang
Shuicheng Yan
ABSTRACT
This work presents a new interactive Content Based Image Retrieval (CBIR) scheme, termed Attribute Feedback (AF). Unlike traditional relevance feedback purely founded on low-level visual features, the Attribute Feedback system shapes users' information needs more precisely and quickly by collecting feedbacks on intermediate level semantic attributes. At each interactive iteration, AF first determines the most informative binary attributes for feedbacks, preferring the attributes that frequently (rarely) appear in current search results but are unlikely (likely) to be users' interest. The binary attribute feedbacks are then augmented by a new type of attributes, "affinity attributes", each of which is off-line learnt to describe the distance between user's envisioned image(s) and a retrieved image with respect to the corresponding affinity attribute. Based on the feedbacks on binary and affinity attributes, the images in corpus are further re-ranked towards better fitting the users' information needs. Extensive experiments on two real-world image datasets well demonstrate the superiority of the proposed scheme over other state-of-the-art relevance feedback based CBIR solutions.
- C. Chang and C. Lin. LIBSVM: A library for support vector machines. ACM TIST, 2011. Google Scholar
Digital Library
- X. Chen, X. Yuan, S. Yan, Y. Rui, and T. Chua. Towards multi-semantic image annotation with graph regularized exclusive group lasso. In MM, 2011. Google ScholarDigital Library
- R. Datta, D. Joshi, J. Li, and J. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 2008. Google ScholarDigital Library
- M. Douze, A. Ramisa, and C. Schmid. Combining attributes and fisher vectors for efficient image retrieval. In CVPR, 2011. Google ScholarDigital Library
- A. Farhadi, I. Endres, and D. Hoiem. Attribute-centric recognition for cross-category generalization. In CVPR, 2010.Google ScholarCross Ref
- A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In CVPR, 2009.Google ScholarCross Ref
- V. Ferrari and A. Zisserman. Learning visual attributes. In NIPS, 2008.Google Scholar
- G. Guo, A. Jain, W. Ma, and H. Zhang. Learning similarity measure for natural image retrieval with relevance feedback. TNN, 2002. Google ScholarDigital Library
- J. Hare, P. Lewis, P. Enser, and C. Sandom. Mind the gap: Another look at the problem of the semantic gap in image retrieval. Multimedia Content Analysis, Management, and Retrieval, 2006.Google Scholar
- T. Huang and X. Zhou. Image retrieval with relevance feedback: From heuristic weight adjustment to optimal learning methods. In ICIP, 2001.Google ScholarCross Ref
- A. Kovashka, D. Parikh, and K. Grauman. Whittlesearch: Image search with relative attribute feedback. In CVPR, 2012. Google ScholarDigital Library
- N. Kumar, A. Berg, P. Belhumeur, and S. Nayar. Describable visual attributes for face verification and image search. TPAMI, 2011. Google ScholarDigital Library
- C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.Google ScholarCross Ref
- T. Leung and J. Malik. Representing and recognizing the visual appearance of materials using three-dimensional textons. IJCV, 2001. Google ScholarDigital Library
- M. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. TOMCCAP, 2006. Google ScholarDigital Library
- D. Parikh and K. Grauman. Relative attributes. In ICCV, 2011. Google ScholarDigital Library
- Y. Rui, T. Huang, and S. Mehrotra. Content-based image retrieval with relevance feedback in mars. In ICIP, 1997.Google ScholarCross Ref
- B. Siddiquie, R. Feris, and L. Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, 2011. Google ScholarDigital Library
- A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. TPAMI, 2000. Google ScholarDigital Library
- S. Tong and E. Chang. Support vector machine active learning for image retrieval. In MM, 2001. Google ScholarDigital Library
- S. Uchihashi and T. Kanade. Content-free image retrieval based on relations exploited from user feedbacks. In ICME, 2005.Google ScholarCross Ref
- A. Vedaldi and B. Fulkerson. Vlfeat: An open and portable library of computer vision algorithms. In MM, 2010. Google ScholarDigital Library
- M. Wang and X. Hua. Active learning in multimedia annotation and retrieval: A survey. TIST, 2011. Google ScholarDigital Library
- Z.-J. Zha., L. Yang, T. Mei, M. Wang, and Z.-F. Wang. Visual query suggestion. In MM, 2009. Google ScholarDigital Library
- W. Zhou., Y. LU, H. Li, Y. Song, and Q. Tian. Spatial Coding for Large Scale Partial-Duplicate Web Image Search. In MM, 2010. Google ScholarDigital Library
- M. Wang, K. Yang, X.-S. Hua, and H. Zhang. Towards a relevant and diverse search of social images. TMM, 2010. Google ScholarDigital Library
- X. Wang, K. Liu, and X. Tang. Query-specific visual semantic spaces for web image re-ranking. In CVPR, 2011.Google ScholarDigital Library
- J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. Huang, and S. Yan. Sparse representation for computer vision and pattern recognition. PIEEE, 2010.Google ScholarCross Ref
- Z.-J. Zha, M. Wang, Y.-T. Zheng, Y. Yang, R. Hong, and T.-S. Chua. Interactive video indexing with statistical active learning. TMM, 2012. Google ScholarDigital Library
- F. Wu, Y. Han, Q. Tian, and Y. Zhuang. Multi-label boosting for image annotation by structural grouping sparsity. In MM, 2010. Google ScholarDigital Library
- Y. Yang, Y. Yang, Z. Huang, H. Shen, and F. Nie. Tag localization with spatial correlations and joint group sparsity. In CVPR, 2011. Google ScholarDigital Library
- J. Yuan, Z.-J. Zha, Y. Zheng, M. Wang, X. Zhou, and T.-S. Chua. Utilizing related samples to enhance interactive concept-based video search. TMM, 2011. Google ScholarDigital Library
- Z.-J. Zha., L. Yang, T. Mei, M. Wang, Z.-F. Wang, T.-S. Chua, and X.-S. Hua. Visual query suggestion: Towards capturing user intent in internet image search. TOMCCAP, 2010. Google ScholarDigital Library
- X. Zhou and T. Huang. Small sample learning during multimedia retrieval using biasmap. In CVPR, 2001.Google Scholar
- X. Zhou and T. Huang. Relevance feedback in image retrieval: A comprehensive review. Multimedia Systems, 2003.Google Scholar
Index Terms
- Attribute feedback
Recommendations
Attribute feedback
MM '12: Proceedings of the 20th ACM international conference on MultimediaThis demonstration presents a new interactive Content Based Image Retrieval (CBIR) system, termed Attribute Feedback (AF). Unlike traditional relevance feedback purely founded on low-level features, AF system shapes user's search intents more precisely ...
Improving video event retrieval by user feedback
In content based video retrieval videos are often indexed with semantic labels (concepts) using pre-trained classifiers. These pre-trained classifiers (concept detectors), are not perfect, and thus the labels are noisy. Additionally, the amount of pre-...
The effect of low-level image features on pseudo relevance feedback
Relevance feedback (RF) is a technique popularly used to improve the effectiveness of traditional content-based image retrieval systems. However, users must provide relevant and/or irrelevant images as feedback for their queries, which is a tedious ...
Comments