Abstract
The automatic registration of different remote sensing images of the same scene is an important problem in the processing of remote sensing images. Such images usually have different resolutions and computing an accurate registration of them in real time is thus often a challenging task. In this paper, a new statistical approach is developed for the accurate and efficient registration of two remote sensing images with different resolutions. The proposed approach utilizes a statistical model to evaluate the probability for each possible mapping between the two images and computes the one with the maximum probability for registration. Similar to most of the state-of-the-art methods for remote sensing image registration, the proposed approach assumes the existence of an affine transformation between the images to be registered. The registration is performed in three stages. In the first stage, the pixels in each image are efficiently partitioned into two sets with the Otsu’s algorithm and four approximate equations for the parameters in the affine transformation are established. In the second stage, pixels in both images with significant edge geometric features are selected for potential pixel matching. In the third stage, based on the four approximate equations generated in the first stage, an Markov Chain Model-based method is used to efficiently compute the matching that can pair the selected pixels with the maximum likelihood. The parameters of the affine transformation can be determined from the pixel pairs with a least square regression approach. Experimental results on a number of pairs of remote sensing images show that this approach can generate registration results more accurate than those obtained with a few state-of-the-art approaches. In addition, real-time evaluation also shows that the approach is computationally efficient and can be used in real-time applications for remote sensing image processing.
Similar content being viewed by others
References
Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: A survey. IEEE Trans. Med. Imag. 32, 1153–1190 (2013)
Feyisa, G.L., Meilby, H., Fensholt, R., Proud, S.R.: Automated water extraction index: a new technique for surface water mapping using landsat imagery. Remote Sens. Environ. 140, 23–25 (2014)
Leprince, S., Barbot, S., Ayoub, F., Avouac, J.P.: Automatic and precise orthorectification, coregistration, and subpixel correlation of satellite images, application to ground deformation measurements. IEEE Trans. Geosci. Remote Sens. 45, 1529–1558 (2007)
Ma, J., Zhao, J., Ma, Y., Tian, J.: Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recognit. 48, 772–784 (2015)
Wu, B., Zhang, Y.S., Zhu, Q.: Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations. ISPRS J. Photogramm. Remote Sens. 68, 40–55 (2012)
Chen, T., Wen, G., Jiang, Z., Yin, B.: Edge feature matching of remote sensing images via parameter decomposition of affine transformation model. ISPRS Ann Photogram Remote Sens Spatial Inform Sci 1–7, 267–272 (2012)
Ling, X., Zhang, Y., Xiang, J., Huang, X., Chen, Z.: An image matching algorithm integrating global SRTM and image segmentation for multi-source satellite imagery. Remote Sens. 8, 672 (2016)
Li, K., Zhang, Y., Zhang, Z., Lai, J.: A coarse-to-fine registration strategy for multi-sensor images with large resolution differences. Remote Sens. 11, 470 (2019)
Peng, Z., Wu, J., Zhang, Y., Lin, X.: A high-speed feature matching method of high-resolution aerial images. J. Real Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-01012-8
De Lima, R., Cabrera-Ponce, A.A., Martinez-Carranza, J.: Parallel hashing-based matching for real-time aerial image mosaicing. J. Real Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-00959-y
Bleyer, M., Gelautz, M.: A layered stereo matching algorithm using image segmentation and global visibility constraints. ISPRS J. Photogramm. Remote Sens. 59, 128–150 (2005)
Heipke, C., Oberst, J., Albertz, J., Attwenger, M., Dorninger, P.: Evaluating planetary digital terrain models-The HRSC DTM test. Planet. Space Sci. 55, 2173–2191 (2007)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)
Gruen, A.: Development and status of image matching in photogrammetry. Photogramm. Rec. 27, 36–57 (2012)
Otsu, N.: A threshold selection method from gray level histograms. IEEE Trans. Syst. Man Cybern 9(1), 62–66 (1979)
Wu, B., Zhang, Y.S., Zhu, Q.: A Triangulation-based hierarchical image matching method for wide-baseline images. Photogramm. Eng. Remote Sens. 77, 695–708 (2011a)
Zhang, L. Automatic Digital Surface Model (DSM) Generation from Linear Array Images. Ph.D. Thesis, Swiss Federal Institute of Technology, Zurich, Switzerland, 2005.
IMAGINE AutoSync™ User’s Guide. Available online: https://faculty.une.edu/cas/szeeman/rs/docs/AutoSync.pdf. Accessed Sept 2020
Cao, S., Jiang, J., Zhang, G., et al.: An edge-based scale- and affine-invariant algorithm for remote sensing image registration. Int. J. Remote Sens. 34(7–8), 2301–2326 (2013)
Zhao, L.Y., Lu, B.Y., Li, X.R., et al.: Multi-source remote sensing image registration based on scale-invariant feature transform and optimization of regional mutual information. Acta Physica Sinica 64(12), 1–10 (2015)
Liang, J., Liu, X., Huang, K., Li, X., Wang, D., Wang, X.: Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans. Geosci. Remote Sens. 52, 603–615 (2014)
Wu, B., Zhang, Y.S., Zhu, Q.: A triangulation-based hierarchical image matching method for wide-baseline images. Photogramm. Eng. Remote Sens. 77, 695–708 (2011b)
Liu, Z., An, J., Jing, Y.: A simple and robust feature point matching algorithm based on restricted spatial order constraints for aerial image registration. IEEE Trans. Geosci. Remote Sens. 50, 514–527 (2012)
Long, T.F., Jiao, W.L., He, G.J., Zhang, Z.M.: A fast and reliable matching method for automated georeferencing of remotely-sensed imagery. Remote Sens. 8, 56 (2016)
Silveira M, Feitosa R, Jacobsen K, Brito J, Heckel Y. A Hybrid Method for stereo image matching. In: Proceedings of the the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Beijing, China, 3–11 July 2008; pp. 895–901.
Colerhodes, A., Johnson, K., PLemoigne J, Zavorin I, : Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Trans. Image Process 12, 1495–1511 (2003)
Wong, A., Clausi, D.A.: ARRSI: Automatic registration of remote-sensing images. IEEE Trans. Geosci. Remote Sens. 45, 1483–1493 (2007)
IAlruzouq, R., Habib, A.: semi-automatic registration of multi-source satellite imagery with varying geometric resolutions. Photogramm Eng. Remote Sens 71, 325–332 (2005)
Eugenio, F., Marques, F.: Automatic satellite image georeferencing using a contour-matching approach. IEEE Trans. Geosci. Remote Sens. 41, 2869–2880 (2003)
Chen, M., Shao, Z.F.: Robust affine-invariant line matching for high resolution remote sensing images. Photogramm. Eng. Remote Sens. 79, 753–760 (2013)
Al-Ruzouq, R.I., Al-Zoubi, A., Akawi, E.E., Abueladas, A.A., Niemi, T.M.: Multiple source imagery and linear features for detection of urban expansion in Aqaba City Jordan. Int. J. Remote Sens. 33, 2563–2581 (2012)
Misra I, Moorthi SM, Dhar D, Ramakrishnan R. An automatic satellite image registration technique based on Harris corner detection and Random Sample Consensus (RANSAC) outlier rejection model. In: Proceedings of the 2012 1st International Conference on Recent Advances in Information Technology (RAIT), Dhanbad, India, 15–17 March 2012; pp. 68–73.
Dai, X., Khorram, S.: A feature-based image registration algorithm using improved chain-code representation combined with invariant moments. IEEE Trans. Geosci. Remote Sens. 37, 2351–2362 (1999)
Goncalves, H., Corte-Real, L., Goncalves, J.A.: Automatic image registration through image segmentation and SIFT. IEEE Trans. Geosci. Remote Sens. 49, 2589–2600 (2011)
Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognit. 26, 1277–1294 (1993)
Humenberger M, Engelke T, Kubinger W. A census-based stereo vision algorithm using modified semi-global matching and plane fitting to improve matching quality. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, USA, 13–18 June 2010:77–84.
Ming, H., Jian, J., Zhou, M., Yi, T., Shi, W.: Robust Multisource remote sensing image registration method based on scene shape similarity. Photogram. Eng. Remote Sens. 85(10), 725–736 (2019)
Xiong, X., Xu, Q., Jin, G., Zhang, H., Gao, X.: Rank-based local self-similarity descriptor for optical-to-sar image matching. Lett IEEE Geosci Remote Sens. (2020). https://doi.org/10.1109/LGRS.2019.2955153
Ye, Y., Bruzzone, L., Shan, J., Bovolo, F., Zhu, Q.: Fast and robust matching for multimodal remote sensing image registration. IEEE Trans. Geosci. Remote Sens. 57, 9059–9070 (2019)
Wan, X., Liu, J.G., Li, S., Yan, H.: Phase correlation decomposition: The impact of illumination variation for robust subpixel remotely sensed image matching. IEEE Trans. Geosci. Remote Sens. 57, 6710–6725 (2019)
Yang, H., Li, X., Zhao, L., Chen, S.: A novel coarse-to-fine scheme for remote sensing image registration based on sift and phase correlation. Remote Sensing 11, 1183 (2019)
Ye, Z., et al.: Robust fine registration of multisensor remote sensing images based on enhanced subpixel phase correlation. Sensors 20, 4338 (2020)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)
Leymarie, F., Levine, M.D.: Tracking deformable objects in the plane using an active contour model. IEEE Trans. Pattern Anal. Mach. Intell. 15, 617–634 (1993)
Hirschmüller, H. Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20–25 June 2005; Volume 2: 807–814.
Hirschmüller, H.: Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341 (2008)
Gong, M., Zhao, S., Jiao, L., Tian, D., Wang, S.: A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Trans. Geosci. Remote Sens. 52, 4328–4338 (2014)
Tong, X., Ye, Z., Xu, Y., Liu, S., Li, L., Xie, H., Li, T.: A novel subpixel phase correlation method using singular value decomposition and unified random sample consensus. IEEE Trans. Geosci. Remote Sens. 53, 4143–4156 (2015)
Barbieux, K.: Pushbroom hyperspectral data orientation by combining feature-based and area-based co-registration techniques. Remote Sens. 10, 645 (2018)
Cole-Rhodes, A.A., Johnson, K.L., LeMoigne, J., Zavorin, I.: Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Trans. Image Process. 12, 1495–1511 (2003)
Thévenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Trans. Image Process. 9, 2083–2099 (2000)
Le Moigne, J., Campbell, W.J., Cromp, R.F.: An automated parallel image registration technique based on the correlation of wavelet features. IEEE Trans. Geosci. Remote Sens. 40, 1849–1864 (2002)
Wu, Y., Ma, W., Miao, Q., Wang, S.: Multimodal continuous ant colony optimization for multisensor remote sensing image registration with local search. Swarm Evol. Comput. 7, 1–10 (2017)
Zeng, Q., Adu, J., Liu, J., Yang, J., Xu, Y., Gong, M.: Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT. J. Real-Time Image Proc. 17, 1103–1115 (2020)
Mondal, P., Banerjee, S.: FPGA-accelerated adaptive projection-based image registration. J. Real-Time Image Proc. (2020). https://doi.org/10.1007/s11554-020-00952-5
Nandalike, R., Sarojadevi, H.: Multimodal image feature detection with ROI-based optimization for image registration. J. Real-Time Image Proc. 17, 1007–1013 (2020)
Acknowledgements
The authors are grateful for the constructive comments and suggestions from the editor of the journal and the anonymous reviewers on an earlier version of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Song, Y., Qu, J. & Liu, C. Real-time registration of remote sensing images with a Markov chain model. J Real-Time Image Proc 18, 1527–1540 (2021). https://doi.org/10.1007/s11554-020-01043-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-020-01043-1