ABSTRACT
The vast user-provided image tags on the popular photo sharing websites may greatly facilitate image retrieval and management. However, these tags are often imprecise and/or incomplete, resulting in unsatisfactory performances in tag related applications. In this work, the tag refinement problem is formulated as a decomposition of the user-provided tag matrix D into a low-rank refined matrix A and a sparse error matrix E, namely D = A + E, targeting the optimality measured by four aspects: 1) low-rank: A is of low-rank owing to the semantic correlations among the tags; 2) content consistency: if two images are visually similar, their tag vectors (i.e., column vectors of A) should also be similar; 3) tag correlation: if two tags co-occur with high frequency in general images, their co-occurrence frequency (described by two row vectors of A) should also be high; and 4) error sparsity: the matrix E is sparse since the tag matrix D is sparse and also humans can provide reasonably accurate tags. All these components finally constitute a constrained yet convex optimization problem, and an efficient convergence provable iterative procedure is proposed for the optimization based on accelerated proximal gradient method. Extensive experiments on two benchmark Flickr datasets, with 25K and 270K images respectively, well demonstrate the effectiveness of the proposed tag refinement approach.
- R. Datta, D. Joshi, J. Li, and J. Wang. Image retrieval: ideas, influences, and trends of the new age. ACM Computing Surveys, 2008. Google ScholarDigital Library
- A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. TPAMI, 2000. Google ScholarDigital Library
- C. Wang, F. Jing, L. Zhang, and H. Zhang. Scalable search-based image annotation. Multimedia Systems, 2008.Google Scholar
- X. Li, L. Chen, L. Zhang, W. Ma, and F. Lin. Image annotation by large-scale content-based image retrieval. ACM MM, 2006. Google ScholarDigital Library
- X. Rui, M. Li, Z. Li, W. Ma, and N. Yu. Bipartite graph reinforcement model for web image annotation. ACM MM, 2007. Google ScholarDigital Library
- M. Huiskes and M. Lew. The mir flickr retrieval evaluation. ACM MIR, 2008. Google ScholarDigital Library
- R. Zhao and W. Grosky. Narrowing the semantic gap - improved text-based web document retrieval using visual features. TMM, 2002. Google ScholarDigital Library
- A. Torralba, R. Fergus, and W. Freeman. 80 million tiny images: a large dataset for non-parametric object and scene recognition. TPAMI, 2008. Google ScholarDigital Library
- H. Zhang, A. Berg, M. Maire, and J. Malik. Svm-knn: discriminative nearest neighbor classification for visual category recognition. CVPR, 2006. Google ScholarDigital Library
- Y. Liu, R. Jin, and L. Yang. Semi-supervised multi-label learning by constrained non-negative matrix factorization. AAAI, 2006. Google ScholarDigital Library
- C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. CVPR, 2009.Google ScholarCross Ref
- Y. Jin, L. Khan, L. Wang, and M. Awad. Image annotations by combining multiple evidence and wordnet. ACM MM, 2005. Google ScholarDigital Library
- C. Wang, F. Jing, L. Zhang, and H. Zhang. Content-based image annotation refinement. CVPR, 2007.Google ScholarCross Ref
- J. Jia, N. Yu, X. Rui, and M. Li. Multi-graph similarity reinforcement for image annotation refinement. ICIP, 2008.Google Scholar
- D. Liu, X. Hua, L. Yang, M. Wang, and H. Zhang. Tag ranking. WWW, 2009. Google ScholarDigital Library
- H. Xu, J. Wang, X. Hua, and S. Li. Tag refinement by regularized lda. ACM MM, 2009. Google ScholarDigital Library
- D. Liu, X.-S. Hua, M. Wang, and H.-J. Zhang. Image retagging. In ACM MM, 2010. Google ScholarDigital Library
- E. Candes, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM, (submitted). http://watt.csl.illinois.edu/perceive/matrixrank/Files/RobustPCA.pdf Google ScholarDigital Library
- S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin. Graph embedding and extension: a general framework for dimensionality reduction. TPAMI, 2007. Google ScholarDigital Library
- R. Cilibrasi and P. Vitany. The google similarity distance. TKDE, 2007. Google ScholarDigital Library
- A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences, 2009. Google ScholarDigital Library
- Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. UIUC Technical Report UILU-ENG-09--2214, 2009.Google Scholar
- J. Cai, E. Candes, and Z. Shen. A singular value thresholding algorithm for matrix completion. In preprint.Google Scholar
- T. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng. Nus-wide: A real-world web image database from national university of singapore. CIVR, 2009. Google ScholarDigital Library
- R. Larsen. Lanczos bidiagonalization with partial reorthogonalization. Aarhus University Technical Report DAIMI-PB-357, 1998.Google Scholar
- C. Hsu, C. Chang, and C Lin. A practical guide to support vector classification. http://www.csie.ntu.edu.tw/cjlin/papers/guide/guide.pdf.Google Scholar
Index Terms
- Image tag refinement towards low-rank, content-tag prior and error sparsity
Recommendations
Exploiting user information for image tag refinement
MM '11: Proceedings of the 19th ACM international conference on MultimediaPhoto sharing websites allow users to describe images with freely chosen tags. The user-generated tags not only facilitate the users in sharing and organizing images, but also provide large scale meaningful data for image retrieval and management. ...
Content-based tag processing for Internet social images
Online social media services such as Flickr and Zooomr allow users to share their images with the others for social interaction. An important feature of these services is that the users manually annotate their images with the freely-chosen tags, which ...
Social tag enrichment via automatic abstract tag refinement
PCM'12: Proceedings of the 13th Pacific-Rim conference on Advances in Multimedia Information ProcessingCollaborative image tagging systems, such as Flickr, are very attractive for supporting keyword-based image retrieval, but some social tags of these collaboratively-tagged social images might be imprecise. Some people may use general or high-level words ...
Comments