Abstract
Most existing image retrieval systems search similar images on a given single input, while querying based on multiple images is not a trivial. In this paper, we describe a novel image retrieval paradigm that users could input two images as query to search the images that include the content of the two input images-synchronously. In our solution, the deep CNN feature is extracted from each single query image and then fused as the query feature. Due to the role of the two query images is different and changeable, we propose the FWC (Feature weighting by Clustering), a novel algorithm to weight the two query features. All the CNN features in the whole dataset are clustered and the weight of each query is obtained by the distance to the mutual nearest cluster. The effectiveness of our algorithm is evaluated in PASCAL VOC2007 and Microsoft COCO datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chiang, J.L.: Knowledge-based principal component analysis for image fusion. Appl. Math. 8(1L), 223–230 (2014)
Elkan, C.: Using the triangle inequality to accelerate k-means. In: ICML, vol. 3, pp. 147–153 (2003)
Fernando, B., Tuytelaars, T.: Mining multiple queries for image retrieval: on-the-fly learning of an object-specific mid-level representation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2544–2551 (2013)
Fu, Y., Cao, L., Guo, G., et al.: Multiple feature fusion by subspace learning. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 127–134. ACM (2008)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Ghodrati, A., Jia, X., Pedersoli, M., et al.: Swap retrieval: retrieving images of cats when the query shows a dog. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 395–402. ACM (2015)
Gawande, U., Zaveri, M., Kapur, A.: A novel algorithm for feature level fusion using SVM classifier for multibiometrics-based person identification. Appl. Comput. Intell. Soft Comput. 2013, 9 (2013)
Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 (2014)
Hariharan, B., Arbeláez, P., Girshick, R., et al.: Hypercolumns for object segmentation and fine-grained localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447–456 (2015)
Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Makadia, A.: Feature tracking for wide-baseline image retrieval. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 310–323. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15555-0_23
Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2161–2168. IEEE (2006)
Razavian, A.S., Azizpour, H., Sullivan, J., et al.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)
Vaca-Castano, G., Shah, M.: Semantic image search from multiple query images. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 887–890. ACM (2015)
Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 660–673. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33709-3_47
Yang, X., Qian, X., Xue, Y.: Scalable mobile image retrieval by exploring contextual saliency. IEEE Trans. Image Proc. 24(6), 1709–1721 (2015)
Liu, X., Wang, M., Yin, B.-C., Huet, B., Li, X.: Event-based media enrichment using an adaptive probabilistic hypergraph model. IEEE Trans. Cybern. 45(11), 2461–2471 (2015)
Wang, M., Li, W., Liu, D., Ni, B., Shen, J., Yan, S.: Facilitating image search with a scalable and compact semantic mapping. IEEE Trans. Cybern. 45(8), 1561–1574 (2015)
Wang, M., Li, G., Lu, Z., Gao, Y., Chua, T.-S.: When Amazon meets Google: product visualization by exploring multiple information sources. ACM Trans. Internet Technol. 12(4), 1–17 (2013). Article 12
Wang, M., Gao, Y., Ke, L., Rui, Y.: View-based discriminative probabilistic modeling for 3D object retrieval and recognition. IEEE Trans. Image Proces. 22(4), 1395–1407 (2013)
Acknowledgment
This work was partially supported by National High Technology Research and Development Program of China (Grant No. 2014AA015104), the Natural Science Foundation of China (NSFC) under Grant 61502139 and 61472116, The Natural Science Foundation of Anhui Province under Grant 1608085MF128, and the program from the Key Lab of Information Network Security, Ministry of Public Security under Grant C14605.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Wu, K., Liu, X., Shao, J., Hong, R., Yang, T. (2016). Retrieving Images by Multiple Samples via Fusing Deep Features. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9916. Springer, Cham. https://doi.org/10.1007/978-3-319-48890-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-48890-5_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48889-9
Online ISBN: 978-3-319-48890-5
eBook Packages: Computer ScienceComputer Science (R0)