Skip to main content
SpringerLink
Log in

Sparse semantic metric learning for image retrieval

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Typical content-based image retrieval solutions usually cannot achieve satisfactory performance due to the semantic gap challenge. With the popularity of social media applications, large amounts of social images associated with user tagging information are available, which can be leveraged to boost image retrieval. In this paper, we propose a sparse semantic metric learning (SSML) algorithm by discovering knowledge from these social media resources, and apply the learned metric to search relevant images for users. Different from the traditional metric learning approaches that use similar or dissimilar constraints over a homogeneous visual space, the proposed method exploits heterogeneous information from two views of images and formulates the learning problem with the following principles. The semantic structure in the text space is expected to be preserved for the transformed space. To prevent overfitting the noisy, incomplete, or subjective tagging information of images, we expect that the mapping space by the learned metric does not deviate from the original visual space. In addition, the metric is straightforward constrained to be row-wise sparse with the ℓ2,1-norm to suppress certain noisy or redundant visual feature dimensions. We present an iterative algorithm with proved convergence to solve the optimization problem. With the learned metric for image retrieval, we conduct extensive experiments on a real-world dataset and validate the effectiveness of our approach compared with other related work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://www.flickr.com.

  2. http://picasa.google.com.

  3. http://www.zooomr.com.

  4. In practice, \(\|{\bf m}^{l}\|_2\) could be close to zero but not zero. Theoretically, it could be zeros. For this case, we can regularize \(D_{ll}=\frac{1}{2\sqrt{\|{\bf m}^{l}\|_2^2+\epsilon}}, \) where \(\epsilon\) is very small constant. When \(\epsilon\rightarrow 0, \) we can see that \(\frac{1}{2\sqrt{\|{\bf m}^{l}\|_2^2+\epsilon}}\) approximates \(\frac{1}{2\|{\bf m}^{l}\|_2}. \)

References

  1. Bar-Hillel, A., Weinshall, D.: Learning a mahalanobis metric from equivalence constraints. J. Mach. Learn. Res. 6, 937–965 (2005)

    MathSciNet  MATH  Google Scholar 

  2. Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng Y.: Nus-wide: a real-world web image database from national university of singapore. In: ACM Conference on Image and Video Retrieval (2009)

  3. Cox, T., Cox, M.: Multidimensional scaling. Chapman and Hall, London (2001)

    MATH  Google Scholar 

  4. Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: International Conference on Machine Learning, pp. 209–216 (2007)

  5. Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: ACM Conference on Knowledge Discovery and Data Mining (2001)

  6. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Elsevier, Amsterdam (1990)

    MATH  Google Scholar 

  7. Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Annual Conference on Neural Information Processing Systems NIPS (2005)

  8. Gu, Q., Zhou, J.: Co-clustering on manifolds. In: KDD (2009)

  9. Guan, N., Tao, D., Luo, Z., Shawe-Taylor, J.: Mahnmf: Manhattan non-negative matrix factorization. CoRR abs/1207.3438 (2012)

  10. Henrion, D., Malick, J.: Projection methods in convex optimization. LAAS-CNRS Research Report 10730 (2010)

  11. Hoi, S.C.H., Liu, W., Lyu, M.R., Ma, W.Y.: Learning distance metrics with contextual constraints for image retrieval. In: CVPR (2006)

  12. Huang, K., Ying, Y., Campbell, C.: Gsml: A unified framework for sparse metric learning. In: International Conference on Data Mining, pp. 189–198 (2009)

  13. Long, M., Wang, J., Ding, G., Shen, D., Yang, Q.: Transfer learning with graph co-regularization. In: AAAI (2012)

  14. Nie, F., Huang, H., Cai, X., Ding, C.H.Q.: Efficient and robust feature selection via joint l21-norms minimization. In: Annual Conference on Neural Information Processing Systems (2010)

  15. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10):1345–1359 (2010)

    Article  Google Scholar 

  16. Qi, G.J., Hua, X.S., Zhang, H.J.: Learning semantic distance from community-tagged media collection. In: ACM Multimedia (2009)

  17. Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  18. Schultz, M., Joachims, T.: Learning a distance metric from relative comparisons. In: Advances in Neural Information Processing Systems (2003)

  19. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)

    Article  Google Scholar 

  20. Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  21. Wang, C., Mahadevan, S.: Heterogeneous domain adaptation using manifold alignment. In: IJCAI (2011)

  22. Wang M., Yang K., Hua X.S., Zhang H.J.: Towards a relevant and diverse search of social images. IEEE Trans. Multimedia 12(8), 829–842 (2010)

    Article  Google Scholar 

  23. Weinberger, K., Saul, L.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)

    MATH  Google Scholar 

  24. Wu, L., Hoi, S.C., Jin, R., Zhu, J., Yu, N.: Distance metric learning from uncertain side information with application to automated photo tagging. In: ACM International Conference on Multimedia (2009)

  25. Wu, P., Hoi, S.C., Zhao, P., He, Y.: Mining social images with distance metric learning for automated image tagging. In: ACM International Conference on Web Search and Data Mining (2011)

  26. Xing, E.P., Jordan, M.I., Karp, R.M., Russell, S.J.: Distance metric learning with application to clustering with side information. In: Advances in Neural Information Processing Systems, pp. 1489–1496 (2002)

  27. Yang, L., Jin, R., Sukthankar, R., Liu, Y.: An efficient algorithm for local distance metric learning. In: AAAI (2006)

  28. Yang, Q., Chen, Y., Xue, G., W. Dai, Y.Y.: Heterogeneous transfer learning for image clustering via the social web. In: ACL/AFNLP (2009)

  29. Yang, Y., Shen, H.T., Ma, Z., Huang, Z., Zhou, X.: L21-norm regularized discriminative feature selection for unsupervised learning. In: International Joint Conference on Artificial Intelligence (2011)

  30. Yu, J., Liu, D., D. Tao, H.S.S.: Complex object correspondence construction in two-dimensional animation. IEEE Trans. Image Process. 20(11), 3257–3269 (2011)

  31. Yu, J., Liu, D., Tao, D., Seah, H.S.: On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans. Syst. Man Cybern. Part B 42(5), 1413–1427 (2012)

    Article  Google Scholar 

  32. Yu, J., Tao, D., Rui, Y., Cheng, J.: Pairwise constraints based multiview features fusion for scene classification. Pattern Recognit. 46(2), 483–496 (2013)

    Article  MATH  Google Scholar 

  33. Yu, J., Tao, D., Wang, M.: Adaptive hypergraph learning and its application in image classification. IEEE Trans. Image Process. 21(7), 3262–3272 (2012)

    Article  MathSciNet  Google Scholar 

  34. Yu, J., Wang, M., Tao, D.: Semi-supervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 21(11), 4636–4648 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported by the 973 Program (Project No. 2012CB316304) and the National Natural Science Foundation of China (Grant No. 60833006 and 61272329).

Author information

Authors and Affiliations

  1. National Lab. of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, No. 95, Zhongguancun East Road, Beijing, 100190, People’s Republic of China

    Jing Liu, Zechao Li & Hanqing Lu

Authors
  1. Jing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zechao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanqing Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, J., Li, Z. & Lu, H. Sparse semantic metric learning for image retrieval. Multimedia Systems 20, 635–643 (2014). https://doi.org/10.1007/s00530-013-0308-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-013-0308-2

Keywords

Search

Navigation