Abstract
In this paper, a new method for image mosaicing (image stitching) is introduced based on Scale Invariant Feature transform (SIFT). One of the main drawbacks of SIFT is the redundancy of the extracted keypoints, which leads to lower image mosaicing quality. Recently, a new method called Redundant Keypoint Elimination (RKEM) was presented to remove these redundant features, and enhance image registration performance. Despite the applicability of RKEM, its threshold value is considered the same in all parts of the image. This characteristic leads to inappropriate removal of keypoints due to the fact that distribution of keypoints in the high-detailed region is denser than the low-detailed ones. This paper proposes a new method to improve RKEM called Clustered RKEM (CRKEM) which is based on keypoints distribution. Moreover, in this paper a new blending algorithm is proposed based on a Gaussian-weighted function. In the proposed blending method, the Gaussian function is proposed based on the mean and variance of the pixels in the overlapped region of images to be mosiaced. In comparison with the classical methods, the experimental results confirm the superiority of the proposed method in image mosaicing as well as to image registration and matching.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Margoon Waterfall in Fars Province, Iran.
Uunderground city of Kariz, Kish, Iran.
References
Kekec, T., Yildirim, A., Unel, M.: A new approach to real-time mosaicing of aerial images. Robot. Auton. Syst. 62, 1755–1767 (2014)
Vaghela D, Naina P. A review of image mosaicing techniques. arXiv preprint arXiv:1405.2539, (2014)
Sharma, S.K., Jain, K.: Image stitching using AKAZE features. J. Indian Soc. Remote Sens. 48, 1389–1401 (2020)
Wang, Z., Yang, Z.: Review on image-stitching techniques. Multimed. Syst. 26, 1–18 (2020)
Saha, M., Chakraborty, M., Biswas, T.: An improved approach for document image mosaicing. Int. J. 6, 51–55 (2016)
Kaur, J.: A robust technique for image mosaicing using modified. Indian J. Sci. Technol. (2016). https://doi.org/10.17485/ijst/2016/v9i47/101722
Jinwei, C., Bin, G., Gangxiang, G.: Image registration and mosaicking based on the criterion of four collinear points. DEStech Trans. Eng. Technol. Res. (2016). https://doi.org/10.12783/dtetr/ICMITE20162016/4576
Yan, W., Yue, G., Fang, Y., Chen, H., Tang, C., Jiang, G.: Perceptual objective quality assessment of stereoscopic stitched images. Signal Process. 172, 107541 (2020)
Zhang, Y., Lai, Y.-K., Zhang, F.-L.: Stereoscopic image stitching with rectangular boundaries. Vis. Comput. 35, 823–835 (2019)
Irani, M., Hsu, S., Anandan, P.: Video compression using mosaic representations. Signal Process.: Image Commun. 7, 529–552 (1995)
Hu R, Shi R, Shen I.-F, Chen W. Video stabilization using scale-invariant features. In Information Visualization, 2007. IV'07. 11th International Conference, (2007), pp. 871–877
Okade, M., Biswas, P.K.: Improving video stabilization using multi-resolution MSER features. IETE J. Res. 60, 373–380 (2014)
Niu, C., Zhong, F., Xu, S., Yang, C., Qin, X.: Cylindrical panoramic mosaicing from a pipeline video through MRF based optimization. Vis. Comput. 29, 253–263 (2013)
Choi Y.-H, Seong Y. K, Choi T.-S. Image mosaicing with automatic scene segmentation for video indexing. In: Consumer Electronics, 2002. ICCE. 2002 Digest of Technical Papers. International Conference on, (2002), pp. 74-75
Szeliski R, Shum H-Y. Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 251–258. (1997)
Zhang, T., Zhao, R., Chen, Z.: Application of migration image registration algorithm based on improved SURF in remote sensing image mosaic. IEEE Access 8, 163637–163645 (2020)
Gracias N, Costeira J. P, Victor J. Linear global mosaics for underwater surveying. In 5th IFAC Symposium on Intelligent Autonomous Vehicles, pp. 78–83. (2004)
Zhang X, Zhu X. An accurate and efficient image registration algorithm in the aerial infrared images. In: Eleventh International Conference on Graphics and Image Processing (ICGIP 2019), p. 113730W. (2020)
Deshmukh P, Paikrao P. A review of various image mosaicing techniques. In: 2019 Innovations in Power and Advanced Computing Technologies (i-PACT), pp. 1–4. (2019)
Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends® Comput. Gr. Vis. 2, 1–104 (2006)
Shum, H.-Y., Szeliski, R.: Construction of panoramic image mosaics with global and local alignment. In: Benosman, R., Kang, S.B. (eds.) Panoramic Vision, pp. 227–268. Springer, New York (2001)
Wei, L., Zhong, Z., Lang, C., Yi, Z.: A survey on image and video stitching. Virtual Real. Intell. Hardw. 1, 55–83 (2019)
Jain, P.M., Shandliya, V.K.: A review paper on various approaches for image mosaicing. Int. J. Comput. Eng. Res. 3, 106–109 (2013)
Monali R, Moonka S, Priya A, Tripathy S. S. Effects of noise and relative overlap on image mosaicing using SURF features. In: Recent Trends in Electronics, Information & Communication Technology (RTEICT), IEEE International Conference on, pp. 773–777. (2016)
Adel E, Elmogy M, Elbakry H. Image stitching based on feature extraction techniques: a survey. In: International Journal of Computer Applications (0975–8887) Volume, pp. 1–8, (2014)
Krishnakumar, K., Gandhi, S.I.: Video stitching based on multi-view spatiotemporal feature points and grid-based matching. Vis. Comput. 36, 1837–1846 (2020)
Pandey, A., Pati, U.C.: Panorama generation using feature-based mosaicing and modified graph-cut blending. In: Pant, M., Ray, K., Sharma, T.K., Rawat, S., Bandyopadhyay, A. (eds.) Soft Computing: Theories and Applications. Springer, Singapore (2018)
Mistry, S., Patel, A.: Image stitching using Harris feature detection. Int. Res. J. Eng. Technol. (IRJET) 3, 2220–2226 (2016)
Bheda, D., Joshi, M., Agrawal, V.: A study on features extraction techniques for image mosaicing. Int. J. Innov. Res. Comput. Commun. Eng. 2, 3432–3437 (2014)
Khan, H.A., Haider, M.A., Ansari, H.A., Ishaq, H., Kiyani, A., Sohail, K., et al.: Automated feature detection in dental periapical radiographs by using deep learning. Oral Surg., Oral Med., Oral Pathol. Oral Radiol. 131, 711–720 (2020)
Bhowmik A, Gumhold S, Rother C, Brachmann E. Reinforced feature points: optimizing feature detection and description for a high-level task. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4948–4957. (2020)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)
Derpanis K. G. The harris corner detector," York University, vol. 2, (2004)
Yang, A., Yang, X., Wu, W., Liu, H., Zhuansun, Y.: Research on feature extraction of tumor image based on convolutional neural network. IEEE Access 7, 24204–24213 (2019)
Kamboj, A., Rani, R., Nigam, A.: A comprehensive survey and deep learning-based approach for human recognition using ear biometric. Vis. Comput. (2021). https://doi.org/10.1007/s00371-021-02119-0
Scheidegger, F., Istrate, R., Mariani, G., Benini, L., Bekas, C., Malossi, C.: Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy. Vis. Comput. 37, 1–18 (2020)
Hemanth, D.J., Estrela, V.V.: Deep Learning for Image Processing Applications. IOS Press, Amsterdam (2017)
Jiao, L., Zhao, J.: A survey on the new generation of deep learning in image processing. IEEE Access 7, 172231–172263 (2019)
Joshi, K., Patel, M.I.: Recent advances in local feature detector and descriptor: a literature survey. Int. J. Multimed. Inf. Retr. 9, 1–17 (2020)
Ghosh, D., Kaabouch, N.: A survey on image mosaicing techniques. J. Vis. Commun. Image Represent. 34, 1–11 (2016)
Prathap K. S. V, Jilani S, Reddy P. R A critical review on image mosaicing. In: Computer Communication and Informatics (ICCCI), 2016 International Conference on, pp. 1–8 (2016)
Bhosle, U., Chaudhuri, S., Roy, S.D.: A fast method for image mosaicing using geometric hashing. IETE J. Res. 48, 317–324 (2002)
Vishwakarma, A., Bhuy, M.: Image mosaicking using improved auto-sorting algorithm and local difference-based harris features. Multimed. Tools Appl. 79, 1–18 (2020)
Zagrouba, E., Barhoumi, W., Amri, S.: An efficient image-mosaicing method based on multifeature matching. Mach. Vis. Appl. 20, 139–162 (2009)
Kang P, Ma H. An automatic airborne image mosaicing method based on the SIFT feature matching. In: Multimedia Technology (ICMT), 2011 International Conference on, pp. 155–159. (2011)
Murali, Y., Madanapalle, M.: Image mosaic using speeded up robust feature detection. Image 1, 40–45 (2012)
Prathap, K.S.V., Jilani, S., Reddy, P.R.: A real-time image mosaicing using scale invariant feature transform. Indian J. Sci. Technol. 9, 1–6 (2016)
Hossein-nejad Z, Nasri M. Image registration based on SIFT features and adaptive RANSAC transform. In: Communication and Signal Processing (ICCSP), 2016 International Conference on. pp. 1087–1091. (2016)
Hossein-Nejad, Z., Nasri, M.: An adaptive image registration method based on SIFT features and RANSAC transform. Comput. Electr. Eng. 62, 524–537 (2017)
Hossein-Nejad, Z., Agahi, H., Mahmoodzadeh, A.: Detailed review of the scale invariant feature transform (sift) algorithm; concepts, indices and applications. J. Mach. Vis. Image Process. 7, 165–190 (2020)
Laraqui, A., Baataoui, A., Saaidi, A., Jarrar, A., Masrar, M., Satori, K.: Image mosaicing using voronoi diagram. Multimed. Tools Appl. 76, 8803–8829 (2017)
Laraqui, A., Saaidi, A., Satori, K.: MSIP: multi-scale image pre-processing method applied in image mosaic. Multimed. Tools Appl. 77, 7517–7537 (2018)
Ke Y, Sukthankar R. PCA-SIFT: A more distinctive representation for local image descriptors. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, pp. II-506-II-513 Vol. 2. (2004)
Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer Vision–ECCV 2006, pp. 404–417. Springer, Berlin (2006)
Cheung W, Hamarneh G. N-sift: N-dimensional scale invariant feature transform for matching medical images. In: Biomedical Imaging: From Nano to Macro, 2007. ISBI 2007. 4th IEEE International Symposium on, pp. 720–723. (2007)
Yi, Z., Zhiguo, C., Yang, X.: Multi-spectral remote image registration based on SIFT. Electron. Lett. 44, 107–108 (2008)
Lingua, A., Marenchino, D., Nex, F.: Performance analysis of the SIFT operator for automatic feature extraction and matching in photogrammetric applications. Sensors 9, 3745–3766 (2009)
Morel, J.-M., Yu, G.: ASIFT: A new framework for fully affine invariant image comparison. SIAM J. Imag. Sci. 2, 438–469 (2009)
Tamimi, H., Andreasson, H., Treptow, A., Duckett, T., Zell, A.: Localization of mobile robots with omnidirectional vision using particle filter and iterative sift. Robot. Auton. Syst. 54, 758–765 (2006)
Sedaghat, A., Mokhtarzade, M., Ebadi, H.: Uniform robust scale-invariant feature matching for optical remote sensing images. IEEE Trans. Geosci. Remote Sens. 49, 4516–4527 (2011)
Hossein-Nejad, Z., Nasri, M.: RKEM: redundant keypoint elimination method in image registration. IET Image Proc. 11, 273–284 (2017)
Hossein-Nejad, Z., Nasri, M.: A-RANSAC: adaptive random sample consensus method in multimodal retinal image registration. Biomed. Signal Process. Control 45, 325–338 (2018)
Hossein-Nejad Z, Nasri M. Retinal image registration based on auto-adaptive SIFT and redundant keypoint elimination method. In: 2019 27th Iranian Conference on Electrical Engineering (ICEE), pp. 1294–1297 (2019)
Liu, Y., Yu, D., Chen, X., Li, Z., Fan, J.: TOP-SIFT: the selected SIFT descriptor based on dictionary learning. Vis. Comput. 35, 667–677 (2019)
Hossein-Nejad, Z., Nasri, M.: Copy-move image forgery detection using redundant keypoint elimination method. In: Ramakrishnan, S. (ed.) Cryptographic and Information Security Approaches for Images and Videos, pp. 773–797. CRC Press, Boca Raton (2019)
Yonghong, J.: Fusion of landsat TM and SAR image based on principal component analysis. Remote Sens. Technol. Appl. 13, 46–49 (1998)
Tian F, Shi P. Image mosaic using orb descriptor and improved blending algorithm. In: Image and Signal Processing (CISP), 2014 7th International Congress on, pp. 693–698 (2014)
Chipman L. J, Orr T. M, Graham L. N. Wavelets and image fusion. In: Image Processing, 1995. Proceedings., International Conference on, pp. 248-251 (1995)
Li, H., Manjunath, B., Mitra, S.K.: Multisensor image fusion using the wavelet transform. Gr Models Image Process. 57, 235–245 (1995)
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Gr. (TOG) 2, 217–236 (1983)
Li A, Zhou S, Wang R. An improved method for eliminating ghosting in image stitching. In: 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), pp. 415–418 (2017)
Zhang, Q., Wang, Y., Wang, L.: Registration of images with affine geometric distortion based on maximally stable extremal regions and phase congruency. Image Vis. Comput. 36, 23–39 (2015)
Hossein-Nejad, Z., Agahi, H., Mahmoodzadeh, A.: Image matching based on the adaptive redundant keypoint elimination method in the SIFT algorithm. Pattern Anal. Appl. 24, 669–683 (2020)
Hong J, Lin W, Zhang H, Li L. Image mosaic based on surf feature matching. In: 2009 First International Conference on Information Science and Engineering, pp. 1287–1290 (2009)
Zhen Y, Sun Z, Li J, Peng Y. An airborne remote sensing image mosaic algorithm based on feature points. In: 2016 Sixth International Conference on Instrumentation & Measurement, Computer, Communication and Control (IMCCC), pp. 202–205 (2016)
Zhang, W., Li, X., Yu, J., Kumar, M., Mao, Y.: Remote sensing image mosaic technology based on SURF algorithm in agriculture. J. Image Video Proc. 2018(85), 2018 (2018)
Ai, Y., Kan, J.: Image mosaicing based on improved optimal seam-cutting (January 2020). IEEE Access 8, 181526–181533 (2020)
Ma, W., Wen, Z., Wu, Y., Jiao, L., Gong, M., Zheng, Y., et al.: Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geosci. Remote Sens. Lett. 14, 3–7 (2017)
Tang, H., Pan, A., Yang, Y., Yang, K., Luo, Y., Zhang, S., et al.: Retinal image registration based on robust non-rigid point matching method. J. Med. Imag. Health Inform. 8, 240–249 (2018)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20, 2378–2386 (2011)
Zhang, L., Shen, Y., Li, H.: VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23, 4270–4281 (2014)
Acknowledgements
The authors would like to thank the anonymous reviewers for their valuable comments which improve the quality of the paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
No conflict of interest exists in the submission of this manuscript, and all the authors have agreed to have seen and approved the manuscript for submission.
Human participants or animals
The paper does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hossein-Nejad, Z., Nasri, M. Clustered redundant keypoint elimination method for image mosaicing using a new Gaussian-weighted blending algorithm. Vis Comput 38, 1991–2007 (2022). https://doi.org/10.1007/s00371-021-02261-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-021-02261-9