Skip to main content
Log in

Real-time registration of remote sensing images with a Markov chain model

  • Special Issue paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The automatic registration of different remote sensing images of the same scene is an important problem in the processing of remote sensing images. Such images usually have different resolutions and computing an accurate registration of them in real time is thus often a challenging task. In this paper, a new statistical approach is developed for the accurate and efficient registration of two remote sensing images with different resolutions. The proposed approach utilizes a statistical model to evaluate the probability for each possible mapping between the two images and computes the one with the maximum probability for registration. Similar to most of the state-of-the-art methods for remote sensing image registration, the proposed approach assumes the existence of an affine transformation between the images to be registered. The registration is performed in three stages. In the first stage, the pixels in each image are efficiently partitioned into two sets with the Otsu’s algorithm and four approximate equations for the parameters in the affine transformation are established. In the second stage, pixels in both images with significant edge geometric features are selected for potential pixel matching. In the third stage, based on the four approximate equations generated in the first stage, an Markov Chain Model-based method is used to efficiently compute the matching that can pair the selected pixels with the maximum likelihood. The parameters of the affine transformation can be determined from the pixel pairs with a least square regression approach. Experimental results on a number of pairs of remote sensing images show that this approach can generate registration results more accurate than those obtained with a few state-of-the-art approaches. In addition, real-time evaluation also shows that the approach is computationally efficient and can be used in real-time applications for remote sensing image processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: A survey. IEEE Trans. Med. Imag. 32, 1153–1190 (2013)

    Article  Google Scholar 

  2. Feyisa, G.L., Meilby, H., Fensholt, R., Proud, S.R.: Automated water extraction index: a new technique for surface water mapping using landsat imagery. Remote Sens. Environ. 140, 23–25 (2014)

    Article  Google Scholar 

  3. Leprince, S., Barbot, S., Ayoub, F., Avouac, J.P.: Automatic and precise orthorectification, coregistration, and subpixel correlation of satellite images, application to ground deformation measurements. IEEE Trans. Geosci. Remote Sens. 45, 1529–1558 (2007)

    Article  Google Scholar 

  4. Ma, J., Zhao, J., Ma, Y., Tian, J.: Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recognit. 48, 772–784 (2015)

    Article  Google Scholar 

  5. Wu, B., Zhang, Y.S., Zhu, Q.: Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations. ISPRS J. Photogramm. Remote Sens. 68, 40–55 (2012)

    Article  Google Scholar 

  6. Chen, T., Wen, G., Jiang, Z., Yin, B.: Edge feature matching of remote sensing images via parameter decomposition of affine transformation model. ISPRS Ann Photogram Remote Sens Spatial Inform Sci 1–7, 267–272 (2012)

    Article  Google Scholar 

  7. Ling, X., Zhang, Y., Xiang, J., Huang, X., Chen, Z.: An image matching algorithm integrating global SRTM and image segmentation for multi-source satellite imagery. Remote Sens. 8, 672 (2016)

    Article  Google Scholar 

  8. Li, K., Zhang, Y., Zhang, Z., Lai, J.: A coarse-to-fine registration strategy for multi-sensor images with large resolution differences. Remote Sens. 11, 470 (2019)

    Article  Google Scholar 

  9. Peng, Z., Wu, J., Zhang, Y., Lin, X.: A high-speed feature matching method of high-resolution aerial images. J. Real Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-01012-8

    Article  Google Scholar 

  10. De Lima, R., Cabrera-Ponce, A.A., Martinez-Carranza, J.: Parallel hashing-based matching for real-time aerial image mosaicing. J. Real Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-00959-y

    Article  Google Scholar 

  11. Bleyer, M., Gelautz, M.: A layered stereo matching algorithm using image segmentation and global visibility constraints. ISPRS J. Photogramm. Remote Sens. 59, 128–150 (2005)

    Article  Google Scholar 

  12. Heipke, C., Oberst, J., Albertz, J., Attwenger, M., Dorninger, P.: Evaluating planetary digital terrain models-The HRSC DTM test. Planet. Space Sci. 55, 2173–2191 (2007)

    Article  Google Scholar 

  13. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)

    Article  Google Scholar 

  14. Gruen, A.: Development and status of image matching in photogrammetry. Photogramm. Rec. 27, 36–57 (2012)

    Article  Google Scholar 

  15. Otsu, N.: A threshold selection method from gray level histograms. IEEE Trans. Syst. Man Cybern 9(1), 62–66 (1979)

    Article  MathSciNet  Google Scholar 

  16. Wu, B., Zhang, Y.S., Zhu, Q.: A Triangulation-based hierarchical image matching method for wide-baseline images. Photogramm. Eng. Remote Sens. 77, 695–708 (2011a)

    Article  Google Scholar 

  17. Zhang, L. Automatic Digital Surface Model (DSM) Generation from Linear Array Images. Ph.D. Thesis, Swiss Federal Institute of Technology, Zurich, Switzerland, 2005.

  18. IMAGINE AutoSync™ User’s Guide. Available online: https://faculty.une.edu/cas/szeeman/rs/docs/AutoSync.pdf. Accessed Sept 2020

  19. Cao, S., Jiang, J., Zhang, G., et al.: An edge-based scale- and affine-invariant algorithm for remote sensing image registration. Int. J. Remote Sens. 34(7–8), 2301–2326 (2013)

    Article  Google Scholar 

  20. Zhao, L.Y., Lu, B.Y., Li, X.R., et al.: Multi-source remote sensing image registration based on scale-invariant feature transform and optimization of regional mutual information. Acta Physica Sinica 64(12), 1–10 (2015)

    Google Scholar 

  21. Liang, J., Liu, X., Huang, K., Li, X., Wang, D., Wang, X.: Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans. Geosci. Remote Sens. 52, 603–615 (2014)

    Article  Google Scholar 

  22. Wu, B., Zhang, Y.S., Zhu, Q.: A triangulation-based hierarchical image matching method for wide-baseline images. Photogramm. Eng. Remote Sens. 77, 695–708 (2011b)

    Article  Google Scholar 

  23. Liu, Z., An, J., Jing, Y.: A simple and robust feature point matching algorithm based on restricted spatial order constraints for aerial image registration. IEEE Trans. Geosci. Remote Sens. 50, 514–527 (2012)

    Article  Google Scholar 

  24. Long, T.F., Jiao, W.L., He, G.J., Zhang, Z.M.: A fast and reliable matching method for automated georeferencing of remotely-sensed imagery. Remote Sens. 8, 56 (2016)

    Article  Google Scholar 

  25. Silveira M, Feitosa R, Jacobsen K, Brito J, Heckel Y. A Hybrid Method for stereo image matching. In: Proceedings of the the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Beijing, China, 3–11 July 2008; pp. 895–901.

  26. Colerhodes, A., Johnson, K., PLemoigne J, Zavorin I, : Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Trans. Image Process 12, 1495–1511 (2003)

    Article  MathSciNet  Google Scholar 

  27. Wong, A., Clausi, D.A.: ARRSI: Automatic registration of remote-sensing images. IEEE Trans. Geosci. Remote Sens. 45, 1483–1493 (2007)

    Article  Google Scholar 

  28. IAlruzouq, R., Habib, A.: semi-automatic registration of multi-source satellite imagery with varying geometric resolutions. Photogramm Eng. Remote Sens 71, 325–332 (2005)

    Article  Google Scholar 

  29. Eugenio, F., Marques, F.: Automatic satellite image georeferencing using a contour-matching approach. IEEE Trans. Geosci. Remote Sens. 41, 2869–2880 (2003)

    Article  Google Scholar 

  30. Chen, M., Shao, Z.F.: Robust affine-invariant line matching for high resolution remote sensing images. Photogramm. Eng. Remote Sens. 79, 753–760 (2013)

    Article  Google Scholar 

  31. Al-Ruzouq, R.I., Al-Zoubi, A., Akawi, E.E., Abueladas, A.A., Niemi, T.M.: Multiple source imagery and linear features for detection of urban expansion in Aqaba City Jordan. Int. J. Remote Sens. 33, 2563–2581 (2012)

    Article  Google Scholar 

  32. Misra I, Moorthi SM, Dhar D, Ramakrishnan R. An automatic satellite image registration technique based on Harris corner detection and Random Sample Consensus (RANSAC) outlier rejection model. In: Proceedings of the 2012 1st International Conference on Recent Advances in Information Technology (RAIT), Dhanbad, India, 15–17 March 2012; pp. 68–73.

  33. Dai, X., Khorram, S.: A feature-based image registration algorithm using improved chain-code representation combined with invariant moments. IEEE Trans. Geosci. Remote Sens. 37, 2351–2362 (1999)

    Article  Google Scholar 

  34. Goncalves, H., Corte-Real, L., Goncalves, J.A.: Automatic image registration through image segmentation and SIFT. IEEE Trans. Geosci. Remote Sens. 49, 2589–2600 (2011)

    Article  MATH  Google Scholar 

  35. Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognit. 26, 1277–1294 (1993)

    Article  Google Scholar 

  36. Humenberger M, Engelke T, Kubinger W. A census-based stereo vision algorithm using modified semi-global matching and plane fitting to improve matching quality. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, USA, 13–18 June 2010:77–84.

  37. Ming, H., Jian, J., Zhou, M., Yi, T., Shi, W.: Robust Multisource remote sensing image registration method based on scene shape similarity. Photogram. Eng. Remote Sens. 85(10), 725–736 (2019)

    Article  Google Scholar 

  38. Xiong, X., Xu, Q., Jin, G., Zhang, H., Gao, X.: Rank-based local self-similarity descriptor for optical-to-sar image matching. Lett IEEE Geosci Remote Sens. (2020). https://doi.org/10.1109/LGRS.2019.2955153

    Article  Google Scholar 

  39. Ye, Y., Bruzzone, L., Shan, J., Bovolo, F., Zhu, Q.: Fast and robust matching for multimodal remote sensing image registration. IEEE Trans. Geosci. Remote Sens. 57, 9059–9070 (2019)

    Article  Google Scholar 

  40. Wan, X., Liu, J.G., Li, S., Yan, H.: Phase correlation decomposition: The impact of illumination variation for robust subpixel remotely sensed image matching. IEEE Trans. Geosci. Remote Sens. 57, 6710–6725 (2019)

    Article  Google Scholar 

  41. Yang, H., Li, X., Zhao, L., Chen, S.: A novel coarse-to-fine scheme for remote sensing image registration based on sift and phase correlation. Remote Sensing 11, 1183 (2019)

    Article  Google Scholar 

  42. Ye, Z., et al.: Robust fine registration of multisensor remote sensing images based on enhanced subpixel phase correlation. Sensors 20, 4338 (2020)

    Article  Google Scholar 

  43. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)

    Article  Google Scholar 

  44. Leymarie, F., Levine, M.D.: Tracking deformable objects in the plane using an active contour model. IEEE Trans. Pattern Anal. Mach. Intell. 15, 617–634 (1993)

    Article  Google Scholar 

  45. Hirschmüller, H. Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20–25 June 2005; Volume 2: 807–814.

  46. Hirschmüller, H.: Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341 (2008)

    Article  Google Scholar 

  47. Gong, M., Zhao, S., Jiao, L., Tian, D., Wang, S.: A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Trans. Geosci. Remote Sens. 52, 4328–4338 (2014)

    Article  Google Scholar 

  48. Tong, X., Ye, Z., Xu, Y., Liu, S., Li, L., Xie, H., Li, T.: A novel subpixel phase correlation method using singular value decomposition and unified random sample consensus. IEEE Trans. Geosci. Remote Sens. 53, 4143–4156 (2015)

    Article  Google Scholar 

  49. Barbieux, K.: Pushbroom hyperspectral data orientation by combining feature-based and area-based co-registration techniques. Remote Sens. 10, 645 (2018)

    Article  Google Scholar 

  50. Cole-Rhodes, A.A., Johnson, K.L., LeMoigne, J., Zavorin, I.: Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Trans. Image Process. 12, 1495–1511 (2003)

    Article  MathSciNet  Google Scholar 

  51. Thévenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Trans. Image Process. 9, 2083–2099 (2000)

    Article  MATH  Google Scholar 

  52. Le Moigne, J., Campbell, W.J., Cromp, R.F.: An automated parallel image registration technique based on the correlation of wavelet features. IEEE Trans. Geosci. Remote Sens. 40, 1849–1864 (2002)

    Article  Google Scholar 

  53. Wu, Y., Ma, W., Miao, Q., Wang, S.: Multimodal continuous ant colony optimization for multisensor remote sensing image registration with local search. Swarm Evol. Comput. 7, 1–10 (2017)

    Google Scholar 

  54. Zeng, Q., Adu, J., Liu, J., Yang, J., Xu, Y., Gong, M.: Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT. J. Real-Time Image Proc. 17, 1103–1115 (2020)

    Article  Google Scholar 

  55. Mondal, P., Banerjee, S.: FPGA-accelerated adaptive projection-based image registration. J. Real-Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-00952-5

    Article  Google Scholar 

  56. Nandalike, R., Sarojadevi, H.: Multimodal image feature detection with ROI-based optimization for image registration. J. Real-Time Image Proc. 17, 1007–1013 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful for the constructive comments and suggestions from the editor of the journal and the anonymous reviewers on an earlier version of the paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yinglei Song.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, Y., Qu, J. & Liu, C. Real-time registration of remote sensing images with a Markov chain model. J Real-Time Image Proc 18, 1527–1540 (2021). https://doi.org/10.1007/s11554-020-01043-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01043-1

Keywords

Navigation