Abstract
Fuzzy image retrieval is a novel visual application about designing a multi-modal retrieval system that supports querying across image modalities, e.g., a fuzzy type image searches for some similar images. However, most existing deep hashing methods are not suitable for obtaining a robust image hash code on multi-modal retrieval task. In this paper, we propose Fusing Semantic Prior based Deep Hashing (FSPDH) method, which is the first attempt to integrate unsupervised semantic prior into end-to-end deep architecture for fuzzy image retrieval task. The major contribution in this work is extracting the prior information from images and incorporating it effectively into hash learning process. In addition, our strategy can be usefully used in single-modal retrieval task. Extensive experiments show that our FSPDH approach yields state-of-the-art results in both multi-modal and single-modal image retrieval tasks on our image datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
In this paper, “data point” represents an image-image pair and “sample” represents an image for one modality.
- 4.
- 5.
References
Bu, S., Wang, L., Han, P., Liu, Z., Li, K.: 3D shape recognition and retrieval based on multi-modality deep learning. Neurocomputing 259, 183–193 (2017)
Cao, Z., Long, M., Wang, J., Yang, Q.: Transitive hashing network for heterogeneous multimedia retrieval, pp. 81–87 (2017)
Castrejon, L., Aytar, Y., Vondrick, C., Pirsiavash, H., Torralba, A.: Learning aligned cross-modal representations from weakly aligned data, pp. 2940–2949 (2016)
Ding, G., Guo, Y., Zhou, J., Gao, Y.: Large-scale cross-modality search via collective matrix factorization hashing. IEEE Trans. Image Process. 25(11), 5427–5440 (2016)
Dorfer, M., Schlüter, J., Vall, A., Korzeniowski, F., Widmer, G.: End-to-end cross-modality retrieval with CCA projections and pairwise ranking loss. arXiv preprint arXiv:1705.06979 (2017)
Drai-Zerbib, V., Baccino, T.: The effect of expertise in music reading: cross-modal competence. J. Eye Mov. Res. 6(5) (2014)
Jiang, Q.Y., Li, W.J.: Deep cross-modal hashing. arXiv preprint arXiv:1602.02255 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks, pp. 1097–1105 (2012)
Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks, pp. 3270–3278 (2015)
Li, Y., Wang, R., Huang, Z., Shan, S., Chen, X.: Face video retrieval with image query via hashing across euclidean space and riemannian manifold, pp. 4758–4767 (2015)
Lin, K., Yang, H.F., Hsiao, J.H., Chen, C.S.: Deep learning of binary hash codes for fast image retrieval, pp. 27–35 (2015)
Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval, pp. 2064–2072 (2016)
Liu, H., Ji, R., Wu, Y., Hua, G.: Supervised matrix factorization for cross-modality hashing (2016)
Masci, J., Bronstein, M.M., Bronstein, A.M., Schmidhuber, J.: Multimodal similarity-preserving hashing. IEEE Trans. Pattern Anal. Mach. Intell. 36(4), 824–830 (2014)
Qu, W., Wang, D., Feng, S., Zhang, Y., Yu, G.: A novel cross-modal hashing algorithm based on multimodal deep learning. Sci. China Inf. Sci. 60(9), 092104 (2017)
Song, J., Yang, Y., Yang, Y., Huang, Z., Shen, H.T.: Inter-media hashing for large-scale retrieval from heterogeneous data sources, pp. 785–796 (2013)
Wang, D., Gao, X., Wang, X., He, L., Yuan, B.: Multimodal discriminative binary embedding for large-scale cross-modal retrieval. IEEE Trans. Image Process. 25(10), 4540–4554 (2016)
Wang, W., Ooi, B.C., Yang, X., Zhang, D., Zhuang, Y.: Effective multi-modal retrieval based on stacked auto-encoders. Proc. VLDB Endow. 7(8), 649–660 (2014)
Wang, W., Yang, X., Ooi, B.C., Zhang, D., Zhuang, Y.: Effective deep learning-based multi-modal retrieval. VLDB J. 25(1), 79–101 (2016)
Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing, pp. 1753–1760 (2009)
Wu, B., Yang, Q., Zheng, W.S., Wang, Y., Wang, J.: Quantized correlation hashing for fast cross-modal search, pp. 3946–3952 (2015)
Wu, P., Hoi, S.C., Zhao, P., Miao, C., Liu, Z.Y.: Online multi-modal distance metric learning with application to image retrieval. IEEE Trans. Knowl. Data Eng. 28(2), 454–467 (2016)
Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning, vol. 1, pp. 2156–2162 (2014)
Xia, S., Li, T., Ge, S., Dong, Z.: Efficient web video classification via cross-modality knowledge transferring, pp. 211–216 (2016)
Zhao, F., Huang, Y., Wang, L., Tan, T.: Deep semantic ranking based hashing for multi-label image retrieval, pp. 1556–1564 (2015)
Zhu, X., Huang, Z., Shen, H.T., Zhao, X.: Linear cross-modal hashing for efficient multimedia search, pp. 143–152 (2013)
Acknowledgments
This work was partially supported by Shanghai Municipal Commission of Economy and Informatization (No. 201701052). We thank Zhongyi Zhou from Shanghai Jiao Tong University for his useful discussions and feedback.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Gong, X., Huang, L., Wang, F. (2018). Fusing Semantic Prior Based Deep Hashing Method for Fuzzy Image Retrieval. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11012. Springer, Cham. https://doi.org/10.1007/978-3-319-97304-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-97304-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97303-6
Online ISBN: 978-3-319-97304-3
eBook Packages: Computer ScienceComputer Science (R0)