Skip to main content
SpringerLink
Log in

Diffeomorphic matching with multiscale kernels based on sparse parameterization for cross-view target detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

We present a novel, robust target detection method to locate a target from a reference image (UAV image) according to a target image (remote sensing satellite image). Using sparse parameterization diffeomorphic matching based on multiscale kernels, the approach modeling the nonrigid transformation function between the reference image and target image is proposed to complete target detection. Furthermore, it designs an feature point matching fusing intensity and phase information to determine the corresponding keypoints, which solves the cross-view problem. Then, the displacements of the corresponding keypoint sets are classified into several subsets using the probabilistic mixture model. The sparse parameterization diffeomorphic matching is executed in the subsets, removing the influence of outliers in the corresponding keypoints. The subset with the maximum evaluation for the transformation is utilized to locate the target. Finally, multiscale kernels based on sparse parameterization are integrated into diffeomorphic matching, solving the large deformation problems between target and reference images. The proposed approach incorporates the stationary velocity field into the diffeomorphism and utilizes the Lie group idea for the stationary velocity to trade off the matching accuracy and computational time. On the University-1652 image dataset with multi-view and multisource properties, experimental results show that the proposed approach is robust to noise and large deformations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Zamir A R, Shah M (2010) Accurate image localization based on google maps street view. In: European conference on computer vision, pp 255–268

  2. Amir R Z, Mubarak S (2014) Image geo-localization based on multiplenearest neighbor feature matching usinggeneralized graphs. IEEE Trans Pattern Anal Mach Intell 36(8):1546–1588

    Article  Google Scholar 

  3. Hays J, Efros A (2008) Im2gps: estimating geographic information from a single image. In: in IEEE Conference on computer vision and pattern recognition, pp 1–8

  4. Schindler G, Brown M (2007) City-scale location recognition. In: in IEEE Conference on computer vision and pattern recognition, pp 1–7

  5. Zhu P, Wen L (2018) Vision meets drones: A challenge. In: in European conference on computer vision, pp 437–468

  6. Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization. In: in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5617–5626

  7. Uss M L, Voze B, Lukin VV, Chehdi K (2016) Efficient rotation-scaling-translation parameter estimation based on the fractal image model. IEEE Trans Geosci Remote Sens 54(1):197–212

    Article  Google Scholar 

  8. Zemene E, Tesfaye Y T, Idrees H, Prati A, Pelillo M, Shah M (2019) Large-scale image geo-localization using dominant sets. IEEE Trans Pattern Anal Mach Intell 41(1):148–161. https://doi.org/10.1109/TPAMI.2017.2787132

    Article  Google Scholar 

  9. Chen Y, Zheng W, Lai J, Yuen PC (2017) An asymmetric distance model for cross-view feature mapping in person reidentification. IEEE Trans Circ Syst Video Technol 27(8):1661–1675

    Article  Google Scholar 

  10. Ben X, Gong C, Zhang P, Jia X, Wu Q, Meng W (2019) Coupled patch alignment for matching cross-view gaits. IEEE Trans Image Process 28(6):3142–3157

    Article  MathSciNet  MATH  Google Scholar 

  11. Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5617–5626

  12. Yang Y, Chen Z, Li X, Guan W, Zhong D, Xu M (2020) Robust template matching with large angle localization. Neurocomputing 398:0925–2312

    Article  Google Scholar 

  13. Guo H, Rangarajan A, Joshi S (2006) Diffeomorphic point matching, Handbook of Mathematical Models in Computer Vision, 205–219

  14. Lowe D G (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  15. Tan H, Ngo C (2009) Localized matching using earth mover’s distance towards discovery of common patterns from small image samples. Image Vis Comput 27(10):1470–1483

    Article  Google Scholar 

  16. Aldana-Iuit J C O (2016) In the saddle: Chasing fast and repeatable features In Proceedings of the international conference on pattern recognition, pp 675–680

  17. DeTone D, Malisiewicz T, Rabinovich A (2018) Superpoint: Self-supervised interest point detection and description. In: in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 337–349

  18. Luo Z, Zhou L, Bai X, Chen H, Zhang J, Yao Y, Li S, Fang T, Quan L (2020) Aslfeat: Learning local features of accurate shape and localization. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 6588–6597

  19. Li Z, Mahapatra D, Tielbeek J A, Stoker J (2015) Image registration based on autocorrelation of local structure. IEEE Trans Image Process 35:63–75

    Article  Google Scholar 

  20. Cao S Y, Shen H L, Chen S J, Li C (2020) Boosting structure consistency for multispectral and multimodal image registration. IEEE Trans Image Process 29:5147–5162

    Article  MATH  Google Scholar 

  21. Ma J, Jiang X, Fan A, Jiang J, Yan J (2020) Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, https://doi.org/10.1007/s11263-020-01359-2

  22. Mirza F, Michael M, Alain T, Laurent Y (2005) Computing large deformation metric mapping via geodesic flows of diffeomorphisms. Int J Comput Vis 61:139–157

    Article  MATH  Google Scholar 

  23. Krebs J, Delingette H, Mailhe B, Ayache N, Mansi T (2019) Learning a probabilistic model for diffeomorphic registration. IEEE Trans Med Imaging 38(9):2165–2176

    Article  Google Scholar 

  24. Zhang M, Fletcher P (2015) Finite-dimensional lie algebras for fast diffeomorphic image registration, Springer International Publishing

  25. Ivan K, Jehoon L, Gregory S, Patricio V, Allen T (2016) A stochastic approach to diffeomorphic point set registration with landmark constraints. IEEE Trans Pattern Anal Mach Intell 38:238–251

    Article  Google Scholar 

  26. Vincent AR (2006) Processing data in lie groups: An algebraic approach. application to non-linear registration and diffusion tensor mri. INRIA Sophia-Antipolis Projet de recherche Asclepios, 181–222

  27. Safdari M, Moallem P, Satari M (2019) Sift detector boosted by adaptive contrast threshold to improve matching robustness of remote sensing panchromatic images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(2):675–684. https://doi.org/10.1109/JSTARS.2019.2892360

    Article  Google Scholar 

  28. Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152

    Article  Google Scholar 

  29. Almhdie A, Christophe L, Deriche M, Roger L (2007) 3d registration using a new implementation of the icp algorithm based on a comprehensive lookup matrix: Application to medical imaging. Pattern Recogn Lett 28(12):1523–1533

    Article  Google Scholar 

  30. Dirk H, Thrun S, Burgard W (2003) An extension of the icp algorithm for modeling nonrigid objects with mobile robots. In: in Proceedings of the eighteenth international joint conference on artificial intelligence, pp 9–15

  31. Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: A survey. IEEE Trans Med Imaging 32(7):1153–1190

    Article  Google Scholar 

  32. Arsigny V, Commowick O, Pennec X, Ayache N (2006) A log-euclidean framework for statistics on diffeomorphisms. In: International Conference on Medical Image Computing and Computer-Assisted Intervention

  33. Vercauteren T, Pennec X, Perchant A, Ayache N (2008) Symmetric log-domain diffeomorphic registration: A demons-based approach

  34. Avants B, Epstein C, Grossman M, Gee J (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41

    Article  Google Scholar 

  35. Younes, Laurent (2007) Jacobi fields in groups of diffeomorphisms and applications. Q Appl Math 65(1):113–134

    Article  MathSciNet  MATH  Google Scholar 

  36. Ashburner J, Friston J (2007) Diffeomorphic registration using geodesic shooting and gauss-newton optimisation. NeuroImage 55(3):113–134

    Google Scholar 

  37. Vialard F, Risser L, Rueckert D, Ashburner J, Friston J (2010) Diffeomorphic 3d image registration via geodesic shooting using an efficient adjoint calculation

  38. Singh N, Hinkle J, Joshi S, Fletcher P T (2013) A vector momenta formulation of diffeomorphisms for improved geodesic regression and atlas construction

  39. Ruhnau P, Christoph S (2006) Optical stokes flow estimation: An imaging-based control approach. Exp Fluids 42:61–78

    Article  Google Scholar 

  40. Hernandez M (2018) Band-limited stokes large deformation diffeomorphic metric mapping. IEEE J Biomed Health Inform 23:362–373

    Article  Google Scholar 

  41. Hernandez M (2018) Newton-krylov optimization in pde-constrained diffeomorphic registration parameterized in the space of band-limited vector fields. arXiv:1807.05117

  42. Ashburner J (2007) A fast diffeomorphic image registration algorithm. Neuroimage 38:95–113

    Article  Google Scholar 

  43. Darkner S, Pai A, Liptrot MG, Sporring J (2018) Collocation for diffeomorphic deformations in medical image registration. IEEE Trans Pattern Anal Mach Intell 40:1570–1583

    Article  Google Scholar 

  44. Reaungamornrat S, Silva T, Uneri A, Vogt S (2016) Mind demons: Symmetric diffeomorphic deformable registration of mr and ct for image-guided spine surgery. IEEE Trans Med Imaging 35(11):2413–2424

    Article  Google Scholar 

  45. Bruveris M, Risser L, V. F (2012) Mixture of kernels and iterated semidirect product of diffeomorphisms groups. Siam Journal on Multiscale Modeling and Simulation 10(4):1–21

    MathSciNet  MATH  Google Scholar 

  46. Sommer S, L F, Nielsen M, Pennec X (2013) Sparse multi-scale diffeomorphic registration: the kernel bundle framework. J Math Imaging Vision 46(3):292–308

    Article  MathSciNet  MATH  Google Scholar 

  47. Pai A, Sommer S, Srensen L, Darkner S, Sporring J, Nielsen M (2016) Kernel bundle diffeomorphic image registration using stationary velocity fields and wendland basis functions. IEEE Trans Med Imaging 35(6):1369

    Article  Google Scholar 

  48. Mingzhen T, Anqi Q (2018) Multiscale frame-based kernels for large deformation diffeomorphic metric mapping. IEEE TRANSACTIONS ON MEDICAL IMAGING 37(10):2344–2355

    Article  Google Scholar 

  49. Ren Z, Sun Q (2020) Simultaneous global and local graph structure preserving for multiple kernel clustering, IEEE Transactions on Neural Networks and Learning Systems, PP 99

  50. Ren Z, Yang S X, Sun Q, Wang T (2020) Consensus affinity graph learning for multiple kernel clustering. IEEE Transactions on Cybernetics, PP 99

  51. Rueckert D, Frangi J (2001) Automatic construction of 3d statistical deformation models using non-rigid registration, International Conference on Medical Image Computing and Computer-Assisted Intervention, 77–84

  52. Marsland S, Twining C (2003) Constructing data-driven optimal representations for iterative pairwise non-rigid registration

  53. Vaillant M, Miller MI, Younes A (2004) Statistics on diffeomorphisms via tangent space representations. NeuroImage 23(2004):S161–S169

    Article  Google Scholar 

  54. Commowick O, Stefanescu R, Fillard P (2005) Incorporating statistical measures of anatomical variability in atlas-to-subject registration for conformal brain radiotherapy

  55. Dalca A V, Balakrishnan G, Guttag J (2019) Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med Image Anal 57:226–236

    Article  Google Scholar 

  56. Morrone MC, Owens RA (1987) Feature detection from local energy. Pattern Recogn Lett 6(5):303–313. https://doi.org/10.1016/0167-8655(87)90013-4https://doi.org/10.1016/0167-8655(87)90013-4http://www.sciencedirect.com/science/article/pii/0167865587900134http://www.sciencedirect.com/science/article/pii/0167865587900134

    Article  Google Scholar 

  57. P.D.Kovesi (2003) Phase congruency detects corners and edges. The Australian Pattern Recognition Society Conference APRSC2007:309–318. http://www.oalib.com/references/7047456

    Google Scholar 

  58. Jian B, Vemuri B (2011) Robust point set registration using gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 33(8):1633–1645

    Article  Google Scholar 

  59. Blei D M, Kucukelbir A, Mcauliffe J D (2016) Variational inference: A review for statisticians

  60. Zheng Z, Yunchao W, Yi Y (2020) University-1652: A multi-view multi-source benchmark for drone-based geo-localization. arXiv:2002.12186

  61. (2018) Bing maps, https://www.microsoft.com/en-us/maps

  62. Yan P, Tan Y, Tai Y, Wu D, Hao X (2021) Unsupervised learning framework for interest point detection and description via properties optimization. Pattern Recogn 112(1–2):107808

    Article  Google Scholar 

  63. Rublee E, Rabaud V, Konolige K, Bradski G R (2011November) Orb: an efficient alternative to sift or surf. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, Barcelona, Spain

  64. Hu S, Lee G H (2020) Image-based geo-localization using satellite imagery. Int J Comput Vis 128(5):1205–1219

    Article  Google Scholar 

  65. Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  66. Zhu S, Yang T, Chen C (2021) Vigor: Cross-view image geo-localization beyond one-to-one retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3640–3649

  67. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  68. Wang C Y, Bochkovskiy A, Liao H (2021) Scaled-yolov4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13029–13038

Download references

Acknowledgements

This work is supported by the Interdisciplinary Research Foundation of HIT under Grant No. IR2021104, Research Foundation of Science and Technology on Electro-Optical Information Security Control Laboratory under Grant No. 6142107200209, Aeronautical Science Foundation of China under Grant No. 2019ZC077006. Basic Scientific Research Business Expenses of Provincial Universities in Heilongjiang Province No. 2019-KYYWF-1384, and Excellent Discipline Team project no. JDXKTD-2019008.

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Harbin Institute of Technology, No. 2, Yikuang Street, Nangang District, Harbin, 150001, Heilongjiang, China

    Xiaomin Liu, Jun-Bao Li, Huanyu Liu & Tingting Wang

  2. Information and Electronic Technology Institute, Jiamusi University, No. 148, Xuefu Street, Xiangyang District, Jiamusi, 154002, Heilongjiang, China

    Xiaomin Liu, Donghua Yuan & Huaqi Zhao

  3. Science and Technology on Space Physics Laboratory, China Academy of Launch Vehicle Technology, No. 2, Jingbei East Road, Nanyuan Street, Fengtai District, Beijing, 100076, China

    Kai Xue

Authors
  1. Xiaomin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donghua Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun-Bao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huaqi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huanyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tingting Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun-Bao Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Yuan, D., Xue, K. et al. Diffeomorphic matching with multiscale kernels based on sparse parameterization for cross-view target detection. Appl Intell 53, 9689–9707 (2023). https://doi.org/10.1007/s10489-022-03668-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03668-0

Keywords

Search

Navigation