Skip to main content

Advertisement

Springer Nature Link
Log in

Weakly-supervised Semantic Guided Hashing for Social Image Retrieval

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Hashing has been widely investigated for large-scale image retrieval due to its search effectiveness and computation efficiency. In this work, we propose a novel Semantic Guided Hashing method coupled with binary matrix factorization to perform more effective nearest neighbor image search by simultaneously exploring the weakly-supervised rich community-contributed information and the underlying data structures. To uncover the underlying semantic information from the weakly-supervised user-provided tags, the binary matrix factorization model is leveraged for learning the binary features of images while the problem of imperfect tags is well addressed. The uncovered semantic information enables to well guide the discrete hash code learning. The underlying data structures are discovered by adaptively learning a discriminative data graph, which makes the learned hash codes preserve the meaningful neighbors. To the best of our knowledge, the proposed method is the first work that incorporates the hash code learning, the semantic information mining and the data structure discovering into one unified framework. Besides, the proposed method is extended to one deep approach for the optimal compatibility of discriminative feature learning and hash code learning. Experiments are conducted on two widely-used social image datasets and the proposed method achieves encouraging performance compared with the state-of-the-art hashing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Cao, Yue, Long, Mingsheng, Liu, Bin, & Wang, Jianmin (2018). Deep cauchy hashing for hamming space retrieval. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 1229–1237).

  • Cao, Zhangjie, Long, Mingsheng, Wang, Jianmin, & Yu, Philip S. (2017). Hashnet: Deep learning to hash by continuation. In Proceedings of IEEE International Conference on Computer Vision, (pp. 5609–5618).

  • Dizaji, Kamran Ghasedi, Zheng, Feng, Sadoughi, Najmeh, Yang, Yanhua, Deng, Cheng, & Huang, Heng (2018). Unsupervised deep generative adversarial hashing network. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 3664–3673).

  • Gong, Yunchao, Ke, Qifa, Isard, Michael, & Lazebnik, Svetlana. (2013). A multi-view embedding space for modeling internet images, tags, and their semantics. International Journal of Computer Vision, 106(2), 210–233.

    Google Scholar 

  • Gong, Y., Lazebnik, S., Gordo, A., & Perronnin, F. (2013). Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2916–2929.

    Google Scholar 

  • Gordo, Albert, Almazán, Jon, Revaud, Jérôme, & Larlus, Diane. (2017). End-to-end learning of deep visual representations for image retrieval. International Journal of Computer Vision, 124(2), 237–254.

    MathSciNet  Google Scholar 

  • Guan, Ziyu, Xie, Fei, Zhao, Wanqing, Wang, Xiaopeng, Chen, Long, Zhao, Wei, & Peng, Jinye (2018). Tag-based weakly-supervised hashing for image retrieval. In Proceedings of International Joint Conference on Artificial Intelligence, (pp. 3776–3782).

  • Gui, Jie, & Li, Ping (2018). R2SDH: robust rotated supervised discrete hashing. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, (pp. 1485–1493).

  • Gui, Jie, Liu, Tongliang, Sun, Zhenan, Tao, Dacheng, & Tan, Tieniu. (2018). Fast supervised discrete hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(2), 490–496.

    Google Scholar 

  • Heo, Jae-Pil, Lee, Youngwoon, He, Junfeng, Chang, Shih-Fu, & Yoon, Sung eui (2012). Spherical hashing. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2957–2964).

  • Heo, Jae-Pil, Lee, Youngwoon, He, Junfeng, Chang, Shih-Fu, & Yoon, Sung-Eui. (2015). Spherical hashing: Binary code embedding with hyperspheres. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(11), 2304–2316.

    Google Scholar 

  • Hu, Haifeng, Wang, Kun, Lv, Chenggang, Wu, Jiansheng, & Yang, Zhen. (2019). Semi-supervised metric learning-based anchor graph hashing for large-scale image retrieval. IEEE Transactions on Image Processing, 28(2), 739–754.

    MathSciNet  MATH  Google Scholar 

  • Huiskes, Mark J., & Lew, Michael S. (2008). The mir flickr retrieval evaluation. In Proceedings of ACM International Conference on Multimedia Information Retrieval, (pp. 39–43).

  • Indyk, Piotr, & Motwani, Rajeev (1998). Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of ACM Symposium on Theory of Computing, (pp. 604–613).

  • Ji, Jianqiu, Li, Jianmin, Yan, Shuicheng, Zhang, Bo, & Tian, Qi (2012). Super-bit locality-sensitive hashing. In Proceedings of Advances in Neural Information Processing Systems, (pp. 108–116).

  • Jiang, Qing-Yuan, & Li, Wu-Jun (2018). Asymmetric deep supervised hashing. In Proceedings of AAAI Conference on Artificial Intelligence, (pp. 3342–3349).

  • Jiang, Qing-Yuan, Cui, Xue, & Li, Wu-Jun. (2018). Deep discrete supervised hashing. IEEE Transactions on Image Processing, 27(12), 5996–6009.

    MathSciNet  MATH  Google Scholar 

  • Jin, Lu, Li, Kai, Li, Zechao, Xiao, Fu, Qi, Guo-Jun, & Tang, Jinhui. (2019). Deep semantic-preserving ordinal hashing for cross-modal similarity search. IEEE Transactions on Neural Networks and Learning Systems, 30(5), 1429–1440.

    MathSciNet  Google Scholar 

  • Jin, Lu, Shu, Xiangbo, Li, Kai, Li, Zechao, Qi, Guo-Jun, & Tang, Jinhui. (2019). Deep ordinal hashing with spatial attention. IEEE Transactions on Image Processing, 28(5), 2173–2186.

    MathSciNet  Google Scholar 

  • Kan, Meina, Xu, Dong, Shan, Shiguang, & Chen, Xilin. (2014). Semisupervised hashing via kernel hyperplane learning for scalable image search. IEEE Transactions on Circuits and Systems for Video Technology, 24(4), 704–713.

    Google Scholar 

  • Krizhevsky, Alex, Sutskever, Ilya, & Hinton, Geoffrey E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of Advances in Neural Information Processing Systems, (pages 1106–1114).

  • Kulis, Brian, & Darrell, Trevor (2009). Learning to hash with binary reconstructive embeddings. In Proceedings of Advances in Neural Information Processing Systems, (pp. 1042–1050).

  • Kulis, Brian, & Grauman, Kristen (2009). Kernelized locality-sensitive hashing for scalable image search. In Proceedings of IEEE International Conference on Computer Vision, (pp. 2130–2137).

  • Li, Qi, Sun, Zhenan, He, Ran, & Tan, Tieniu (2017). Deep supervised discrete hashing. In Proceedings of Advances in Neural Information Processing Systems, (pp. 2479–2488).

  • Li, Wu-Jun, Wang, Sheng, & Kang, Wang-Cheng (2016). Feature learning based deep supervised hashing with pairwise labels. In Proceedings of International Joint Conference on Artificial Intelligence, (pp. 1711–1717).

  • Lin, Kevin, Yang, Huei-Fang, Hsiao, Jen-Hao, & Chen, Chu-Song (2015). Deep learning of binary hash codes for fast image retrieval. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, (pp. 4933–4941).

  • Liong, Venice Erin, Lu, Jiwen, Tan, Yap-Peng, & Zhou, Jie (2017). Cross-modal deep variational hashing. In Proceedings of IEEE International Conference on Computer Vision, (pp. 4097–4105).

  • Liong, Venice Erin, Lu, Jiwen, Wang, Gang, Moulin, Pierre, & Zhou, Jie (2015). Deep hashing for compact binary codes learning. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, (pp. 2475–2483).

  • Li, Zechao, & Tang, Jinhui. (2015). Weakly supervised deep metric learning for community-contributed image retrieval. IEEE Transactions on Multimedia, 17(11), 1989–1999.

    Google Scholar 

  • Li, Zechao, & Tang, Jinhui. (2017). Weakly supervised deep matrix factorization for social image understanding. IEEE Transactions on Image Processing, 26(1), 276–288.

    MathSciNet  MATH  Google Scholar 

  • Li, Zechao, Tang, Jinhui, & Mei, Tao. (2019). Deep collaborative embedding for social image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9), 2070–2083.

    Google Scholar 

  • Liu, Haomiao, Wang, Ruiping, Shan, Shiguang, Chen, Xilin (2016). Deep supervised hashing for fast image retrieval. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2064–2072).

  • Liu, Wei, Wang, Jun, Kumar, Sanjiv, & Chang, Shih-Fu (2011). Hashing with graphs. In Proceedings of International Conference on Machine Learning, (pp. 1–8).

  • Liu, Xianglong, Deng, Cheng, Lang, Bo, Tao, Dacheng, & Li, Xuelong. (2016). Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 25(2), 907–919.

    MathSciNet  MATH  Google Scholar 

  • Liu, Xianglong, He, Junfeng, & Chang, Shih-Fu. (2017). Hash bit selection for nearest neighbor search. IEEE Transactions on Image Processing, 26(11), 5367–5380.

    MathSciNet  MATH  Google Scholar 

  • Liu, Xianglong, Li, Zhujin, Deng, Cheng, & Tao, Dacheng. (2017). Distributed adaptive binary quantization for fast nearest neighbor search. IEEE Transactions on Image Processing, 26(11), 5324–5336.

    MathSciNet  MATH  Google Scholar 

  • Mandal, Devraj, Chaudhury, Kunal N., & Biswas, Soma. (2019). Generalized semantic preserving hashing for cross-modal retrieval. IEEE Transactions on Image Processing, 28(1), 102–112.

    MathSciNet  Google Scholar 

  • Raginsky, Maxim, & Lazebnik, Svetlana (2009). Locality-sensitive binary codes from shift-invariant kernels. In Proceedings of Advances in Neural Information Processing Systems, (pp. 1509–1517).

  • Shen, Fumin, Shen, Chunhua, Liu, Wei, & Shen, Heng Tao (2015). Supervised discrete hashing. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 37–45).

  • Shen, Fumin, Shen, Chunhua, Shi, Qinfeng, Anton, van den Hengel, & Tang, Zhenmin (2013). Inductive hashing on manifolds. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 1562–1569).

  • Sohn, Sungryull, Kim, Hyunwoo, & Kim, Junmo. (2017). Uncorrelated component analysis-based hashing. IEEE Transactions on Image Processing, 26(8), 3759–3774.

    MathSciNet  MATH  Google Scholar 

  • Tang, Jinhui, & Li, Zechao. (2018). Weakly supervised multimodal hashing for scalable social image retrieval. IEEE Transactions on on Circuits and Systems for Video Technology, 28(10), 2730–2741.

    Google Scholar 

  • Tang, Jinhui, Lin, Jie, Li, Zechao, & Yang, Jian. (2018). Discriminative deep quantization hashing for face image retrieval. IEEE Transactions on Neural Networks and Learning Systtems, 29(12), 6154–6162.

    Google Scholar 

  • Tang, Jinhui, Li, Zechao, Wang, Meng, & Zhao, Ruizhen. (2015). Neighborhood discriminant hashing for large-scale image retrieval. IEEE Transactions on Image Processing, 24(9), 2827–2840.

    MathSciNet  MATH  Google Scholar 

  • Tang, Jinhui, Li, Zechao, & Zhu, Xiang. (2018). Supervised deep hashing for scalable face image retrieval. Pattern Recognition, 75, 25–32.

    Google Scholar 

  • Tang, Jinhui, Shu, Xiangbo, Li, Zechao, Jiang, Yu-Gang, & Tian, Qi. (2019). Social anchor-unit graph regularized tensor completion for large-scale image retagging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), 2027–2034.

    Google Scholar 

  • Tang, Jinhui, Shu, Xiaongbo, Li, Zechao, Qi, Guo-Jun, & Wang, Jingdong. (2016). Generalized deep transfer networks for knowledge propagation in heterogeneous domains. ACM Transactions on Multimedia Computing Communications and Applications, 12(4s), 1–22.

    Google Scholar 

  • Tang, Jinhui, Shu, Xiangbo, Qi, Guo-Jun, Li, Zechao, Wang, Meng, Yan, Shuicheng, et al. (2017). Tri-clustered tensor completion for social-aware image tag refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(8), 1662–1674.

    Google Scholar 

  • Venkateswara, Hemanth, Eusebio, Jose, Chakraborty, Shayok, & Panchanathan, Sethuraman (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (pp. 5385–5394).

  • Wang, Daixin, Cui, Peng, Ou, Mingdong, & Zhu, Wenwu (2015). Deep multimodal hashing with orthogonal units. In Proceedings of International Joint Conference on Artificial Intelligence, (pp. 2291–2297).

  • Wang, Jun, Kumar, Sanjiv, & Chang, Shih-Fu. (2012). Semi-supervised hashing for large scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2393–2406.

    Google Scholar 

  • Wang, Jingdong, Zhang, Ting, Song, Jingkuan, Sebe, Nicu, & Shen, Heng Tao. (2018). A survey on learning to hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 769–790.

    Google Scholar 

  • Weiss, Yair, Torralba, Antonio, & Fergus, Robert (2008). Spectral hashing. In Proceedings of Advances in Neural Information Processing Systems, (pp. 1753–1760).

  • Wu, Fei, Yu, Zhou, Yi, Tang, Siliang, Zhang, & Yin, & Zhuang, Yueting., (2014). Sparse multi-modal hashing. IEEE Transactions on Multimedia, 16(2), 424–439.

  • Xia, Rongkai, Pan, Yan, Lai, Hanjiang, Liu, Cong, & Yan, Shuicheng (2014). Supervised hashing for image retrieval via image representation learning. In Proceedings of AAAI Conference on Artificial Intelligence, (pp. 2156–2162).

  • Zhai, Deming, Liu, Xianming, Ji, Xiangyang, Zhao, Debin, Satoh, Shin’ichi, & Gao, Wen. (2018). Supervised distributed hashing for large-scale multimedia retrieval. IEEE Transactions on Multimedia, 20(3), 675–686.

    Google Scholar 

  • Zhang, Dell, Wang, Jun, Cai, Deng, & Lu, Jinsong (2012). Self-taught hashing for fast similarity search. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval, (pp. 18–25).

  • Zhang, Dongqing, & Li, Wu-Jun (2014). Large-scale supervised multimodal hashing with semantic correlation maximization. In Proceedings of AAAI Conference on Artificial Intelligence, (pp. 143–152).

  • Zhang, Zhongyuan, Li, Tao, Ding, Chris H. Q., & Zhang, Xiang-Sun (2007). Binary matrix factorization with applications. In Proceedings of IEEE International Conference on Data Mining, (pp. 391–400).

  • Zhang, Ziming, Chen, Yuting, & Saligrama, Venkatesh (2016). Efficient training of very deep neural networks for supervised hashing. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, (pp. 1487–1495).

  • Zhang, Haofeng, Liu, Li, Long, Yang, & Shao, Ling. (2018). Unsupervised deep hashing with pseudo labels for scalable image retrieval. IEEE Transactions on Image Processing, 27(4), 1626–1638.

    MathSciNet  Google Scholar 

  • Zhu, Han, Long, Mingsheng, Wang, Jianmin, & Cao, Yue (2016). Deep hashing network for efficient similarity retrieval. In Proceedings of AAAI Conference on Artificial Intelligence, (pp. 2415–2421).

  • Zhu, Xiaofeng, Huang, Zi, Shen, Heng Tao, & Zhao, Xin (2013). Linear cross-modal hashing for efficient multimedia search. In Proceedings of ACM International Conference on Multimedia, (pp. 143–152).

  • Zhuang, Yueting, Liu, Yang, Wu, Fei, Zhang, Yin, & Shao, Jian (2011). Hypergraph spectral hashing for similarity search of social image. In Proceedings of ACM International Conference on Multimedia, (pp. 1457–1460).

Download references

Acknowledgements

This work was partially supported by the National Key R&D Program of China under Grant 2018AAA0102002, the National Natural Science Foundation of China (Grant No. 61772275, 61732007, 61772268 and 61672285) and the Natural Science Foundation of Jiangsu Province (Grant BK20170033).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Zechao Li, Jinhui Tang & Jian Yang

  2. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Liyan Zhang

Authors
  1. Zechao Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jinhui Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Liyan Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jian Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jinhui Tang.

Additional information

Communicated by Li Liu, Matti Pietikäinen, Jie Qin, Jie Chen, Wanli Ouyang, Luc Van Gool.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Tang, J., Zhang, L. et al. Weakly-supervised Semantic Guided Hashing for Social Image Retrieval. Int J Comput Vis 128, 2265–2278 (2020). https://doi.org/10.1007/s11263-020-01331-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-020-01331-0

Keywords