Abstract
In our previous study, to address the prevalent issue of scarcity of labeled remote sensing (RS) images, we proposed a novel RS image retrieval model called similarity based Siamese convolutional neural network (SBS-CNN). SBS-CNN accomplishes unsupervised transfer learning with the help of several pretrained CNNs called CNN experts, which compute similarities for RS image pairs and provide the basis for unsupervised training. In this paper, we further investigate two problems: 1) Is CNN experts’ “opinion” on image similarity the same as humans’? 2) can human opinion help increase retrieval performance? To this end, we propose an effective method of collecting human views, and incorporate them into unsupervised training process by introducing an adaptive “push-pull” mechanism into triplet networks. Experimental results reveal that 1) CNN experts’ views on similarity are quite different from humans’, however, two kinds of views are strongly positively correlated; 2) human views can greatly improve retrieval performance.
Supported by National Natural Science Foundation of China under Grant 61673184.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Aptoula, E.: Remote sensing image retrieval with global morphological texture descriptors. IEEE Trans. Geosci. Remote Sens. 52(5), 3023–3034 (2014)
Dai, O.E., Demir, B., Sankur, B., Bruzzone, L.: A novel system for content-based retrieval of single and multi-label high-dimensional remote sensing images. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 11(7), 2473–2490 (2018)
Demir, B., Bruzzone, L.: Hashing-based scalable remote sensing image search and retrieval in large archives. IEEE Trans. Geosci. Remote Sens. 54(2), 892–904 (2016)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of International Conference Machine Learning, Lille, France, pp. 1180–1189, July 2015
Ge, Y., Jiang, S., Xu, Q., Jiang, C., Ye, F.: Exploiting representations from pre-trained convolutional neural networks for high-resolution remote sensing image retrieval. Multimedia Tools Appl. 77(13), 17489–17515 (2017). https://doi.org/10.1007/s11042-017-5314-5
Li, A., Lu, Z., Wang, L., Xiang, T., Wen, J.R.: Zero-shot scene classification for high spatial resolution remote sensing images. IEEE Trans. Geosci. Remote Sens. 55(7), 4157–4167 (2017)
Li, P., Ren, P., Zhang, X., Wang, Q., Zhu, X., Wang, L.: Region-wise deep feature representation for remote sensing images. Remote Sens. 10(6), 871 (2018)
Li, Y., Zhang, Y., Tao, C., Zhu, H.: Content-based high-resolution remote sensing image retrieval via unsupervised feature learning and collaborative affinity metric fusion. Remote Sens. 8(9), 709 (2016)
Liu, Y., Ding, L., Chen, C., Liu, Y.: Similarity-based unsupervised deep transfer learning for remote sensing image retrieval. IEEE Trans. Geosci. Remote Sens. (2020). https://doi.org/10.1109/TGRS.2020.2984703
Liu, Y., Huang, C.: Scene classification via triplet networks. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 11(1), 220–237 (2018)
Napoletano, P.: Visual descriptors for content-based retrieval of remote sensing images. Int. J. Remote Sens. 39(5), 1343–1376 (2018)
Othman, E., Bazi, Y., Melgani, F., Alhichri, H., Alajlan, N., Zuair, M.: Domain adaptation network for cross-scene classification. IEEE Trans. Geosci. Remote Sens. 55(8), 4441–4456 (2017)
Shao, Z., Yang, K., Zhou, W.: Performance evaluation of single-label and multi-label remote sensing image retrieval using a dense labeling dataset. Remote Sens. 10(6), 964 (2018)
Wang, Y., et al.: A three-layered graph-based learning approach for remote sensing image retrieval. IEEE Trans. Geosci. Remote Sens. 54(10), 6020–6034 (2016)
Xia, G.S., Tong, X.Y., Hu, F., Zhong, Y., Datcu, M., Zhang, L.: Exploiting deep features for remote sensing image retrieval: a systematic investigation. CoRR abs/1409.1556, July 2017. http://arxiv.org/abs/1707.07321
Xiao, Z., Long, Y., Li, D., Wei, C., Tang, G., Liu, J.: High-resolution remote sensing image retrieval based on CNNs from a dimensional perspective. Remote Sens. 9(7), 725 (2017)
Yang, J., Liu, J., Dai, Q.: An improved bag-of-words framework for remote sensing image retrieval in large-scale image databases. Int. J. Digit. Earth 8(4), 273–292 (2015)
Yang, Y., Newsam, S.: Geographic image retrieval using local invariant features. IEEE Trans. Geosci. Remote Sens. 51(2), 818–832 (2013)
Ye, F., Luo, W., Dong, M., He, H., Min, W.: SAR image retrieval based on unsupervised domain adaptation and clustering. IEEE Geosci. Remote Sens. Lett. 16(9), 1482–1486 (2019)
Ye, F., Xiao, H., Zhao, X., Dong, M., Luo, W., Min, W.: Remote sensing image retrieval using convolutional neural network features and weighted distance. IEEE Geosci. Remote Sens. Lett. 15(10), 1535–1539 (2018)
Zhou, W., Newsam, S., Li, C., Shao, Z.: Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval. Remote Sens. 9(5), 489 (2017)
Zhou, W., Newsam, S., Li, C., Shao, Z.: PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS J. Photogramm. Remote Sens. 145, 197–209 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Chen, C., Han, Z., Liu, Y., Ding, L. (2020). Incorporating Human Views into Unsupervised Deep Transfer Learning for Remote Sensing Image Retrieval. In: Lu, Y., Vincent, N., Yuen, P.C., Zheng, WS., Cheriet, F., Suen, C.Y. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2020. Lecture Notes in Computer Science(), vol 12068. Springer, Cham. https://doi.org/10.1007/978-3-030-59830-3_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-59830-3_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59829-7
Online ISBN: 978-3-030-59830-3
eBook Packages: Computer ScienceComputer Science (R0)