Skip to main content
SpringerLink
Log in

Heterogeneous image retrieval based on structural information

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The retrieval of images captured by different sensors or under different conditions is a desirable work. For this task, local image descriptor is an important tool. Nevertheless, most of local descriptors are proposed for similar images, which are usually captured by the same sensor in similar condition so they are same in radiation, or there are only linear radiation changes. In this paper, we propose a novel local feature descriptor, Histogram of Orientated Edge and Texture (HOET), for the retrieval of images with radiometric changes, especially for nonlinear radiation changes. In order to increase the similarity of the images captured from the same scene with nonlinear radiation changes, we combine edge and texture features to describe structural information of the images. First, we use Sobel operators in different directions to construct edge response maps and extract edge description in spatial domain. Second, we use log-Gabor filters to extract texture feature, then construct maximum index map (MIM) and extract texture description in frequency domain. Finally, HOET is developed by combining edge and texture description. We evaluate our descriptor in two heterogeneous image datasets and experiments show that HOET outperforms the state-of-the-art methods, including hand-designed and CNN-based descriptors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aguilera, C.A., Aguilera, F.J., Sappa, A.D., Aguilera, C., Toledo, R.: Learning cross-spectral similarity measures with deep convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 267–275 (2016)

  2. Aguilera, C.A., Sappa, A.D., Toledo, R.: Lghd: A feature descriptor for matching across non-linear intensity variations. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 178–181 (2015)

  3. Balntas, V., Johns, E., Tang, L., Mikolajczyk, K.: Pn-net: Conjoined triple deep network for learning local image descriptors. arXiv:1603.09114 (2016)

  4. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer Vision—ECCV 2006, pp. 404–417. Springer, Berlin Heidelberg, Berlin, Heidelberg (2006)

    Chapter  Google Scholar 

  5. Bracewell, R.: The Fourier Transform and Its Applications, 3rd edn. McGraw-Hill, New York (1999)

    MATH  Google Scholar 

  6. Brown, M., Süsstrunk, S.: Multi-spectral sift for scene category recognition. In: CVPR 2011, pp. 177–184 (2011)

  7. Cabon, Y., Murray, N., Humenberger, M.: Virtual kitti 2. arXiv:2001.10773 (2020)

  8. Chandrasekhar, V., Chen, D., Tsai, S., Cheung, N.M., Chen, H., Takacs, G., Reznik, Y., Vedantham, R., Grzeszczuk, R., Bach, J., Girod, B.: The stanford mobile visual search dataset. In: ACM Multimedia Systems Conference (2011)

  9. Cristhian, A., Angel, S., Cristhian, A.: Cross-spectral local descriptors via quadruplet network. Sensors 17, 4 (2017)

    Google Scholar 

  10. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) vol. 1, pp. 886–893 (2005)

  11. Dellinger, F., Delon, J., Gousseau, Y., Michel, J., Tupin, F.: Sar-sift: A sift-like algorithm for sar images. IEEE Trans. Geosci. Remote Sens. 53(1), 453–466 (2015)

    Article  Google Scholar 

  12. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: Self-supervised interest point detection and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2018)

  13. Fischer, S., Šroubek, F., Perrinet, L., Redondo, R., Cristóbal, G.: Self-invertible 2d log-gabor wavelets. Int. J. Comput. Vision 75, 231–246 (2007)

    Article  Google Scholar 

  14. Fu, Z., Qin, Q., Luo, B., Wu, C., Sun, H.: A local feature descriptor based on combination of structure and texture information for multispectral image matching. IEEE Geosci. Remote Sens. Lett. 16(1), 100–104 (2019)

    Article  Google Scholar 

  15. Kovesi, P.: Phase congruency detects corners and edges. Digital Image Computing: Techniques and Applications pp. 309–318 (2003)

  16. Li, J., Hu, Q., Ai, M.: Rift: Multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Trans. Image Process. 29, 3296–3310 (2020)

    Article  Google Scholar 

  17. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)

    Article  Google Scholar 

  18. Morrone, M.C., Owens, R.A.: Feature detection from local energy. Pattern Recogn. Lett. 6(5), 303–313 (1987)

    Article  Google Scholar 

  19. Mouats, T., Aouf, N., Sappa, A.D., Aguilera, C., Toledo, R.: Multispectral stereo odometry. IEEE Trans. Intell. Transp. Syst. 16(3), 1210–1224 (2015)

    Article  Google Scholar 

  20. Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: Orb-slam: A versatile and accurate monocular slam system. IEEE Trans. Rob. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  21. Ono, Y., Trulls, E., Fua, P., Yi, K.M.: Lf-net: Learning local features from images. In: Advances in neural information processing systems, pp. 6234–6244 (2018)

  22. Oppenheim, A.V., Lim, J.S.: The importance of phase in signals. Proc. IEEE 69(5), 529–541 (1981)

    Article  Google Scholar 

  23. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  24. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer Vision - ECCV 2006, pp. 430–443. Springer, Berlin Heidelberg, Berlin, Heidelberg (2006)

    Chapter  Google Scholar 

  25. Saleem, S., Sablatnig, R.: A robust sift descriptor for multispectral images. IEEE Signal Process. Lett. 21(4), 400–403 (2014)

    Article  Google Scholar 

  26. Shen, X., Wang, C., Li, X., Yu, Z., Li, J., Wen, C., Cheng, M., He, Z.: Rf-net: An end-to-end image matching network based on receptive field. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8124–8132 (2019)

  27. Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F.: Discriminative learning of deep convolutional feature point descriptors. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 118–126 (2015)

  28. Simonyan, K., Vedaldi, A., Zisserman, A.: Learning local feature descriptors using convex optimisation. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1573–1585 (2014)

    Article  Google Scholar 

  29. Sivic, Z.: Video google: a text retrieval approach to object matching in videos. In: Proceedings Ninth IEEE International Conference on Computer Vision vol. 2, pp. 1470–1477 (2003)

  30. Vural, M.F., Yardimci, Y., Temizel, A.: Registration of multispectral satellite images with orientation-restricted sift. In: 2009 IEEE International Geoscience and Remote Sensing Symposium, vol. 3, pp. III–243–III–246 (2009)

  31. Ye, Y., Shan, J., Bruzzone, L., Shen, L.: Robust registration of multimodal remote sensing images based on structural similarity. IEEE Trans. Geosci. Remote Sens. 55(5), 2941–2958 (2017)

    Article  Google Scholar 

  32. Yi, K.M., Trulls, E., Lepetit, V., Fua, P.: Lift: Learned invariant feature transform. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision - ECCV 2016, pp. 467–483. Springer International Publishing, Cham (2016)

  33. Yi, Z., Zhiguo, C., Yang, X.: Multi-spectral remote image registration based on sift. Electron. Lett. 44(2), 107–108 (2008)

  34. Zheng, L., Yang, Y., Hauptmann, A.G.: Person re-identification: Past, present and future. arXiv:1610.02984 (2016)

Download references

Author information

Authors and Affiliations

  1. The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, China

    Hansi Weng, Jun Liu & Bin Luo

Authors
  1. Hansi Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weng, H., Liu, J. & Luo, B. Heterogeneous image retrieval based on structural information. SIViP 16, 1117–1125 (2022). https://doi.org/10.1007/s11760-021-02061-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02061-7

Keywords

Search

Navigation