Skip to main content
Springer Nature Link
Log in

New UAV image registration method based on geometric constrained belief propagation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image registration is a crucial step in the field of computer vision. However, the traditional scale invariant feature transform (SIFT) based method often suffers from many mismatches when being utilized to register unmanned aerial vehicle (UAV) images with obvious rotation, viewpoint change and similar textures. In order to tackle this problem, we formulate the image matching problem as a Markov Random Field (MRF) energy minimization problem and propose an accurate and fast image registration framework. First, a SIFT algorithm is utilized to extract keypoints and an algorithm called “Center of Mass” (CoM) is exploited to assign a single orientation for every point instead of SIFT since SIFT may cause ambiguity in subsequent process; second, based on the local adjacent spatial relationship between a feature point and its neighboring points, a new concept of local spatial constraints is proposed to characterize the geometric consistency between them and incorporated into a belief propagation algorithm to obtain initial matching results; finally the residual mismatches are eliminated by the Random Sample Consensus (RANSAC) algorithm, meanwhile the transformation parameters are estimated. Experiment results demonstrate that the proposed framework can significantly enhance the registration performance in UAV image registration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Aguilar W, Martinez-Perez ME, Frauel Y, Escolano F, Lozano MA, Espinosa-Romero A (2007) Graph-based methods for retinal mosaicing and vascular characterization. In: International workshop on graph-based representations in pattern recognition, Springer, pp 25–36

  2. Aguilar W, Frauel Y, Escolano F, Martinez-Perez ME, Espinosa-Romero A, Lozano MA (2009) A robust graph transformation matching for non-rigid registration. Image Vis Comput 27(7):897–910. https://doi.org/10.1016/j.imavis.2008.05.004

    Article  Google Scholar 

  3. Argyriou V, Vlachos T (2003) Estimation of sub-pixel motion using gradient cross-correlation. Electron Lett 39(13):980–982

    Article  Google Scholar 

  4. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  5. Brown LG (1992) A survey of image registration techniques. ACM Comput Surv (CSUR) 24(4):325–376

    Article  Google Scholar 

  6. Bulatov D, Rottensteiner F, Schulz K (2012) Context-based urban terrain reconstruction from images and videos. In: Proceedings of the XXII ISPRS congress of the international society for photogrammetry and remote sensing ISPRS annals, Melbourne, p 3

  7. Colomina I, Molina P (2014) Unmanned aerial systems for photogrammetry and remote sensing: a review. ISPRS J Photogramm Remote Sens 92:79–97

    Article  Google Scholar 

  8. Evangelidis GD, Psarakis EZ (2008) Parametric image alignment using enhanced correlation coefficient maximization. IEEE Trans Pattern Anal Mach Intell 30(10):1858–1865

    Article  Google Scholar 

  9. Feng S, Deng L, Shu G, Wang F, Deng H, Ji K (2012) A subpixel registration algorithm for low PSNR images. In: 2012 I.E. Fifth International Conference on Advanced Computational Intelligence (ICACI), IEEE, pp 626–630

  10. Gauglitz S, Turk M, Höllerer T (2011) Improving keypoint orientation assignment. In: BMVC, pp 1–11

  11. Guizar-Sicairos M, Thurman ST, Fienup JR (2008) Efficient subpixel image registration algorithms. Opt Lett 33(2):156–158

    Article  Google Scholar 

  12. Ke Y, Sukthankar R (2004) PCA-SIFT: a more distinctive representation for local image descriptors. In: CVPR 2004. Proceedings of the 2004 I.E. computer society conference on computer vision and pattern recognition, 2004, IEEE, pp II–II

  13. Kolmogorov V (2006) Convergent tree-reweighted message passing for energy minimization. IEEE Trans Pattern Anal Mach Intell 28(10):1568–1583

    Article  Google Scholar 

  14. Kupfer B, Netanyahu NS, Shimshoni I (2015) An efficient SIFT-based mode-seeking algorithm for sub-pixel registration of remotely sensed images. IEEE Geosci Remote Sens Lett 12(2):379–383. https://doi.org/10.1109/lgrs.2014.2343471

    Article  Google Scholar 

  15. Li B, Ye H (2012) RSCJ: robust sample consensus judging algorithm for remote sensing image registration. IEEE Geosci Remote Sens Lett 9(4):574–578

    Article  MathSciNet  Google Scholar 

  16. Lowe DG (1999) Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on computer vision, 1999, IEEE, pp 1150–1157

  17. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  18. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481. https://doi.org/10.1109/tgrs.2015.2441954

    Article  Google Scholar 

  19. Zeng T, Yang W, Wu H UAV remote sensing image processing and classification in Wenchuan earthquake district. In: Sixth International Symposium on Multispectral Image Processing and Pattern Recognition, 2009. SPIE, p 6. https://doi.org/10.1117/12.832957

  20. Morel J-M, Yu G (2009) ASIFT: a new framework for fully affine invariant image comparison. SIAM J Imag Sci 2(2):438–469. https://doi.org/10.1137/080732730

    Article  MathSciNet  MATH  Google Scholar 

  21. Qiaoliang L, Guoyou W, Jianguo L, Shaobo C (2009) Robust scale-invariant feature matching for remote sensing image registration. IEEE Geosci Remote Sens Lett 6(2):287–291. https://doi.org/10.1109/lgrs.2008.2011751

    Article  Google Scholar 

  22. Q-s C, Defrise M, Deconinck F (1994) Symmetric phase-only matched filtering of Fourier-Mellin transforms for image registration and recognition. IEEE Trans Pattern Anal Mach Intell 16(12):1156–1168

    Article  Google Scholar 

  23. Salvo G, Caruso L, Scordo A (2014) Urban traffic analysis through an UAV. Procedia Soc Behav Sci 111:1083–1091. https://doi.org/10.1016/j.sbspro.2014.01.143

    Article  Google Scholar 

  24. senseFly dataset. https://www.sensefly.com/drones/example-datasets.html

  25. Shi X, Jiang J (2015) Point-matching method for remote sensing images with background variation. J Appl Remote Sens 9(1):095046–095046

    Article  Google Scholar 

  26. Szeliski R, Zabih R, Scharstein D, Veksler O, Kolmogorov V, Agarwala A, Tappen M, Rother C (2006) A comparative study of energy minimization methods for markov random fields. Comput Vis ECCV 2006:16–29

    Google Scholar 

  27. Tagare HD, Rao M (2015) Why does mutual-information work for image registration? A deterministic explanation. IEEE Trans Pattern Anal Mach Intell 37(6):1286–1296

    Article  Google Scholar 

  28. Tola E, Lepetit V, Fua P (2010) Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans Pattern Anal Mach Intell 32(5):815–830

    Article  Google Scholar 

  29. USC-SIPI dataset. http://sipi.usc.edu/database/database.php

  30. Uss ML, Vozel B, Dushepa VA, Komjak VA, Chehdi K (2014) A precise lower bound on image subpixel registration accuracy. IEEE Trans Geosci Remote Sens 52(6):3333–3345

    Article  Google Scholar 

  31. Wainwright MJ, Jordan MI (2008) Graphical models, exponential families, and variational inference. Found Trends Mach Learn 1(1–2):1–305

    MATH  Google Scholar 

  32. Wainwright M, Jaakkola T, Willsky A (1998) MAP estimation via agreement on (hyper) trees: message-passing and linear programming approaches. In: Proceedings of the annual allerton conference on communication control and computing, 2002. vol 3. The University, pp 1565–1575

  33. Weiss Y, Freeman WT (2001) On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans Inf Theory 47(2):736–744

    Article  MathSciNet  MATH  Google Scholar 

  34. Xu Q, Zhang Y, Li B (2014) Improved SIFT match for optical satellite images registration by size classification of blob-like structures. Remote Sens Lett 5(5):451–460. https://doi.org/10.1080/2150704x.2014.917774

    Article  Google Scholar 

  35. Yang L, Tian Z, Zhao W, Wen J, Yan W (2016) Robust image registration using adaptive coherent point drift method. J Appl Remote Sens 10(2):025014–025014

    Article  Google Scholar 

  36. Yi Z, Zhiguo C, Yang X (2008) Multi-spectral remote image registration based on SIFT. Electron Lett 44(2):107. https://doi.org/10.1049/el:20082477

    Article  Google Scholar 

  37. Yue W, Wenping M, Maoguo G, Linzhi S, Licheng J (2015) A novel point-matching algorithm based on fast sample consensus for image registration. IEEE Geosci Remote Sens Lett 12(1):43–47. https://doi.org/10.1109/lgrs.2014.2325970

    Article  Google Scholar 

  38. Zhou F, Yang W, Liao Q (2012) A coarse-to-fine subpixel registration method to recover local perspective deformation in the application of image super-resolution. IEEE Trans Image Process 21(1):53–66

    Article  MathSciNet  MATH  Google Scholar 

  39. Zhou C, Zhou JH, Fan DH (2014) The study of a fast sub-pixel registration method for remote sensing image. In: Advanced materials research, trans tech publ, pp 3877–3880

  40. Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000. https://doi.org/10.1016/s0262-8856(03)00137-9

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their constructive comments to this paper.

Author information

Authors and Affiliations

  1. School of Remote Sensing and information Engineering, Wuhan University, No. 129, Luoyu Road, Wuhan, 430070, China

    Shuangming Zhao, Guorong Yu & Yunfan Cui

Authors
  1. Shuangming Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Guorong Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yunfan Cui
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Guorong Yu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, S., Yu, G. & Cui, Y. New UAV image registration method based on geometric constrained belief propagation. Multimed Tools Appl 77, 24143–24163 (2018). https://doi.org/10.1007/s11042-018-5727-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5727-9

Keywords