Abstract
Content-based image retrieval (CBIR) has attracted increasing attention in the field of computer-aided diagnosis, for which learning-based hashing approaches represent the most prominent techniques for large-scale image retrieval. In this work, we propose a Supervised Hashing method with Energy-Based Modeling (SH-EBM) for scalable multi-label image retrieval, where concurrence of multiple symptoms with subtle differences in visual feature makes the search problem quite challenging, even for sophisticated hashing models built upon modern deep architectures. In addition to similarity-preserving ranking loss, multi-label classification loss is often employed in existing supervised hashing to further improve the expressiveness of hash codes, by optimizing a normalized probabilistic objective with tractable likelihood (e.g., multi-label cross-entropy). On the contrary, we present a multi-label EBM loss without restriction on the tractability of the log-likelihood, which is more flexible to parameterize and can model a more expressive probability distribution over multimorbidity image data. We further develop a multi-label Noise Contrastive Estimation (ml-NCE) algorithm for discriminative training of the proposed hashing network. On a multimorbidity dataset constructed by the NIH Chest X-ray, our SH-EBM outperforms most supervised hashing methods by a significant margin, implying its effectiveness in facilitating multilevel similarity preservation for scalable image retrieval.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Li, Z., Zhang, X., Müller, H., Zhang, S.: Large-scale retrieval for medical image analytics: a comprehensive review. Med. Image Anal. 43, 66–84 (2018)
Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. Int. J. Med. Informat. 73(1), 1–23 (2004)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: CVPR, pp. 3462–3471 (2017)
Wang, J., Zhang, T., Sebe, N., Shen, H.T.: A survey on learning to hash. TPAMI 40(4), 769–790 (2017)
Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: NIPS, pp. 1753–1760 (2008)
Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. TPAMI 35(12), 2916–2929 (2013)
Wang, J., Kumar, S., Chang, S.: Semi-supervised hashing for large-scale search. TPAMI 34(12), 2393–2406 (2012)
Shen, F., Shen, C., Liu, W., Shen, H.T.: Supervised Discrete Hashing. In: CVPR, pp. 37–45 (2015)
Liong, V.E., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hashing for compact binary codes learning. In: CVPR, pp. 2475–2483 (2015)
Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: CVPR, pp. 2064–2072 (2016)
Cao, Z., Long, M., Wang, J., Yu, P.S.: HashNet: deep learning to hash by continuation. In: ICCV, pp. 5608–5617 (2017)
Conjeti, S., Paschali, M., Katouzian, A., Navab, N.: Deep multiple instance hashing for scalable medical image retrieval. In: MICCAI, pp. 550–558 (2017)
Conjeti, S., Roy, A.G., Katouzian, A., Navab, N.: Hashing with residual networks for image retrieval. In: MICCAI, pp. 541–549 (2017)
Chen, Z., Cai, R., Lu, J., Lu, J., Feng, J., Zhou, J.: Order-sensitive deep hashing for multimorbidity medical image retrieval. In: MICCAI, pp. 620–628 (2018)
Lin, K., Yang, H.F., Hsiao, J.H., Chen, C.S.: Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 27–35 (2015)
Li, Q., Sun, Z., He, R., Tan, T.: Deep supervised discrete hashing. In: NIPS, pp. 2479–2488 (2017)
Yang, H., Lin, K., Chen, C.: Supervised learning of semantics-preserving hash via deep convolutional neural networks. TPAMI 40(2), 437–451 (2018)
Liu, L., Rahimpour, A., Taalimi. A, Qi, H.: End-to-end binary representation learning via direct binary embedding. In: ICIP, pp. 1257–1261 (2017)
Rodrigues, J., Cristo, M., Colonna, J.G.: Deep hashing for multi-label image retrieval: a survey. Artif. Intell. Rev. 53(7), 5261–5307 (2020). https://doi.org/10.1007/s10462-020-09820-x
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., Huang, F.: A tutorial on energy-based learning. Predicting Struc. Data 1 (2006)
Gutmann, M., Hyvärinen, A.: Noise-contrastive estimation: a new estimation principle for unnormalized statistical models. In: AISTATS, pp. 297–304 (2010)
Song, Y., Kingma, D.P.: How to train your energy-based models. arXiv preprint arXiv:2101.03288 (2021)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NIPS, pp. 8024–8035 (2019)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under grants 61972046, and in part by the Beijing Natural Science Foundation under grants 4202051.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, P., Zhou, X., Wei, Z., Guo, G. (2021). Energy-Based Supervised Hashing for Multimorbidity Image Retrieval. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12905. Springer, Cham. https://doi.org/10.1007/978-3-030-87240-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-87240-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87239-7
Online ISBN: 978-3-030-87240-3
eBook Packages: Computer ScienceComputer Science (R0)
