research-article

Deep Priority Hashing

Authors:

Mingsheng Long,

Philip S. YuAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 1653 - 1661

https://doi.org/10.1145/3240508.3240543

Published: 15 October 2018 Publication History

Abstract

Deep hashing enables image retrieval by end-to-end learning of deep representations and hash codes from training data with pairwise similarity information. Subject to the distribution skewness underlying the similarity information, most existing deep hashing methods may underperform for imbalanced data due to misspecified loss functions. This paper presents Deep Priority Hashing (DPH), an end-to-end architecture that generates compact and balanced hash codes in a Bayesian learning framework. The main idea is to reshape the standard cross-entropy loss for similarity-preserving learning such that it down-weighs the loss associated to highly-confident pairs. This idea leads to a novel priority cross-entropy loss, which prioritizes the training on uncertain pairs over confident pairs. Also, we propose another priority quantization loss, which prioritizes hard-to-quantize examples for generation of nearly lossless hash codes. Extensive experiments demonstrate that DPH can generate high-quality hash codes and yield state-of-the-art image retrieval results on three datasets, ImageNet, NUS-WIDE, and MS-COCO.

References

[1]

Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Philip S. Yu. 2017. HashNet: Deep Learning to Hash by Continuation. In The IEEE International Conference on Computer Vision (ICCV).

[2]

Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. 2009. NUS-WIDE: A Real-World Web Image Database from National University of Singapore. In ICMR. ACM.

[3]

Jacek P Dmochowski, Paul Sajda, and Lucas C Parra. 2010. Maximum likelihood in cost-sensitive learning: Model specification, approximations, and upper bounds. Journal of Machine Learning Research (JMLR), Vol. 11, Dec (2010), 3313--3332.

Digital Library

[4]

J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In ICML.

Digital Library

[5]

Venice Erin Liong, Jiwen Lu, Gang Wang, Pierre Moulin, and Jie Zhou. 2015. Deep Hashing for Compact Binary Codes Learning. In CVPR. IEEE, 2475--2483.

[6]

D. J. Fleet, A. Punjani, and M. Norouzi. 2012. Fast search in Hamming space with multi-index hashing. In CVPR. IEEE.

Digital Library

[7]

Aristides Gionis, Piotr Indyk, Rajeev Motwani, et almbox. 1999. Similarity search in high dimensions via hashing. VLDB, Vol. 99. ACM, 518--529.

Digital Library

[8]

Yunchao Gong and Svetlana Lazebnik. 2011. Iterative quantization: A procrustean approach to learning binary codes. In CVPR. 817--824.

Digital Library

[9]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. CVPR (2016).

[10]

H. Jegou, M. Douze, and C. Schmid. 2011. Product Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 33, 1 (Jan 2011), 117--128.

Digital Library

[11]

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. In ACM Multimedia Conference. ACM.

Digital Library

[12]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In NIPS.

Digital Library

[13]

Brian Kulis and Trevor Darrell. 2009. Learning to hash with binary reconstructive embeddings. In NIPS. 1042--1050.

Digital Library

[14]

Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous Feature Learning and Hash Coding with Deep Neural Networks. In CVPR. IEEE.

[15]

Michael S. Lew, Nicu Sebe, Chabane Djeraba, and Ramesh Jain. 2006. Content-based Multimedia Information Retrieval: State of the Art and Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 2, 1 (Feb. 2006), 1--19.

Digital Library

[16]

Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2016. Feature learning based deep supervised hashing with pairwise labels. In IJCAI.

Digital Library

[17]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. 2017. Focal Loss for Dense Object Detection. In The IEEE International Conference on Computer Vision (ICCV).

[18]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In ECCV. Springer, 740--755.

[19]

Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. CVPR. 2064--2072.

[20]

Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In CVPR. IEEE.

[21]

Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2011. Hashing with Graphs. In ICML. ACM.

Digital Library

[22]

Xianglong Liu, Junfeng He, Cheng Deng, and Bo Lang. 2014. Collaborative hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2139--2146.

Digital Library

[23]

Xianglong Liu, Junfeng He, Bo Lang, and Shih-Fu Chang. 2013. Hash bit selection: a unified solution for selection problems in hashing. In CVPR. IEEE, 1570--1577.

Digital Library

[24]

Mohammad Norouzi and David M Blei. 2011. Minimal loss hashing for compact binary codes. In ICML. ACM, 353--360.

Digital Library

[25]

Mohammad Norouzi, David M Blei, and Ruslan R Salakhutdinov. 2012. Hamming distance metric learning. In NIPS. 1061--1069.

Digital Library

[26]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), Vol. 115, 3 (2015), 211--252.

Digital Library

[27]

Ruslan Salakhutdinov and Geoffrey E Hinton. 2007. Learning a nonlinear embedding by preserving class neighbourhood structure. In AISTATS. 412--419.

[28]

Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015. Supervised Discrete Hashing. In CVPR. IEEE.

[29]

K. Simonyan and A. Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR, 2015 (arXiv:1409.1556v6).

[30]

Arnold WM Smeulders, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 22, 12 (2000), 1349--1380.

Digital Library

[31]

Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2012. Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 34, 12 (2012), 2393--2406.

Digital Library

[32]

Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, and Heng Tao Shen. 2018. A Survey on Learning to Hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, 4 (Feb. 2018), 769--790.

[33]

Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral Hashing. In NIPS.

[34]

Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised hashing for image retrieval via image representation learning. In AAAI. AAAI, 2156--2162.

Digital Library

[35]

Peichao Zhang, Wei Zhang, Wu-Jun Li, and Minyi Guo. 2014. Supervised hashing with latent factor models. In SIGIR. ACM, 173--182.

Digital Library

[36]

Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. 2016. Deep Hashing Network for Efficient Similarity Retrieval. In AAAI. AAAI.

Digital Library

Cited By

Fu TZhan YZhang CLuo XChen ZWang YYang XXu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)FedCAFE: Federated Cross-Modal Hashing with Adaptive Feature EnhancementProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681319(9670-9679)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681319
Wang HYang XSun JZhang SChen CHua XLuo X(2024)Look Into Gradients: Learning Compact Hash Codes for Out-of-Distribution RetrievalIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.342526836:12(8730-8743)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3425268
Jing PSun HNie LLi YSu Y(2024)Deep Multi-Modal Hashing With Semantic Enhancement for Multi-Label Micro-Video RetrievalIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333707736:10(5080-5091)Online publication date: Oct-2024
https://doi.org/10.1109/TKDE.2023.3337077
Show More Cited By

Index Terms

Deep Priority Hashing
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Image search

Recommendations

Deep Triplet Quantization
MM '18: Proceedings of the 26th ACM international conference on Multimedia

Deep hashing establishes efficient and effective image retrieval by end-to-end learning of deep representations and hash codes from similarity data. We present a compact coding solution, focusing on deep learning to quantization approach that has shown ...
Partial-Softmax Loss based Deep Hashing
WWW '21: Proceedings of the Web Conference 2021

Recently, deep supervised hashing methods have shown state-of-the-art performance by integrating feature learning and hash codes learning into an end-to-end network to generate high-quality hash codes. However, it is still a challenge to learn ...
Deep Asymmetric Pairwise Hashing
MM '17: Proceedings of the 25th ACM international conference on Multimedia

Recently, deep neural networks based hashing methods have greatly improved the multimedia retrieval performance by simultaneously learning feature representations and binary hash functions. Inspired by the latest advance in the asymmetric hashing scheme,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Key R&D Program of China

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
288
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fu TZhan YZhang CLuo XChen ZWang YYang XXu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)FedCAFE: Federated Cross-Modal Hashing with Adaptive Feature EnhancementProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681319(9670-9679)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681319
Wang HYang XSun JZhang SChen CHua XLuo X(2024)Look Into Gradients: Learning Compact Hash Codes for Out-of-Distribution RetrievalIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.342526836:12(8730-8743)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3425268
Jing PSun HNie LLi YSu Y(2024)Deep Multi-Modal Hashing With Semantic Enhancement for Multi-Label Micro-Video RetrievalIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333707736:10(5080-5091)Online publication date: Oct-2024
https://doi.org/10.1109/TKDE.2023.3337077
Wang HJiang HSun JZhang SChen CHua XLuo X(2024)DIOR: Learning to Hash With Label Noise Via Dual Partition and Contrastive LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.331210936:4(1502-1517)Online publication date: Apr-2024
https://doi.org/10.1109/TKDE.2023.3312109
Zhang FChen CHua XLuo X(2024)FATE: Learning Effective Binary Descriptors With Group FairnessIEEE Transactions on Image Processing10.1109/TIP.2024.340613433(3648-3661)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3406134
Yang XWang HSun JXiao YXiang WChen CHua XLuo X(2024)ROSE: Relational and Prototypical Structure Learning for Universal Domain Adaptive HashingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.344431919(7690-7704)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3444319
Han JWu J(2024)Remote Sensing Image Anti-Interference Matching Algorithm Based on Deep Hashing2024 14th International Symposium on Antennas, Propagation and EM Theory (ISAPE)10.1109/ISAPE62431.2024.10840910(01-04)Online publication date: 23-Oct-2024
https://doi.org/10.1109/ISAPE62431.2024.10840910
Miniadou KLeonidis APapadopoulos GStephanidis C(2024)Encrypted Biometric Search: A Deep Learning Approach to Scalable and Secure Cross-Border Data Exchange2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825332(2794-2800)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10825332
Zhao WSong JYuan SGao LYang YShen HEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Precise Target-Oriented Attack against Deep Hashing-based RetrievalProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612364(6379-6389)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612364
Jiang YXie HZhang LLi PZhang DZhang YEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Dual Dynamic Proxy Hashing Network for Long-tailed Image RetrievalProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612328(8942-8953)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612328
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten