Abstract
Aligning images is one of the main goals of image stitching. Limited by matching accuracy and deformability, the results of current mainstream stitching approaches in large parallax scenes usually contain obvious stitching errors. A misalignment-eliminated warping image stitching based on grid-based motion statistics (GMS) matching is proposed. A matching approach from coarse-to-fine composed of oriented FAST and rotated BRIEF (ORB) and GMS is integrated to provide more accurate and higher number of inliers for subsequent warping. The local homography with global similarity transformation constraint is used to warp the images to achieve initial image alignment. For the projection biases after local warping, a post-processing step based on thin plate spline (TPS) is proposed for further correction. Both qualitative and quantitative comparisons in experiments of challenging cases show that this method can accurately align images while maintaining a natural look at the same time.







Similar content being viewed by others
References
Bastug E, Bennis M, Medard M, Debbah M (2017) Toward interconnected virtual reality: opportunities, challenges, and enablers. IEEE Commun Mag 55(6):110–117
Bian JW, Lin WY, Matsushita Y et al (2017) GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 2828–2837
Bookstein FL (1989) Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell 11(6):567–585
Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73
Caesar H, Bankiti V, Lang AH et al (2020) Nuscenes: a multimodal dataset for autonomous driving. In: Proc IEEE/CVF Conf Comput Vis pattern Recogn, pp 11621–11631
Cao Q, Shi Z, Wang P et al (2020) A seamless image-stitching method based on human visual discrimination and attention. F Appl Sci (Basel) 10(4):1462
Chang CH, Sato Y, Chuang YY (2014)Shape-preserving half projective warps for image stitching. In: Proc IEEE Conf. Comput Vis Pattern Recogn, pp 3254–3261
Chen YS, Chuang YY (2016) Natural image stitching with the global similarity prior. In: Proc Eur Conf Comput Vis, pp 186–201
Debabrata G, Kaabouch N (2016) A survey on image mosaicing techniques. J Vis Commun Image Represent 34:1–11
Edward R, Drummond T (2006) Machine learning for high-speed corner detection. In: Comput Vis ECCV
Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun Assoc Comp Mach 24(6):381–395
Gao J, Kim SJ, Brown MS (2011) Constructing image panoramas using dual-homography warping. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 49–56
Guo Y, Zhao R, Wu S, Wang C (2018) Image capture pattern optimization for panoramic photography. Multimed Tools Appl 77(17):22299–22318
Han J, Pauwels EJ, De Zeeuw P (2013) Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recogn Lett 34(1):42–51
Hardy RL (1990) Theory and applications of the multiquadric-biharmonic method 20 years of discovery 1968–1988. Comput Math Appl 19(8–9):163–208
Harris C, Stephens M (1988) A combined corner and edge detector. In: Proceedings Alvey vision Conference, pp 147–151
Hejazifar H, Hassan K (2018) Fast and robust seam estimation to seamless image stitching. Signal Image Video Process 12(5):885–893
Krishnakumar K, Gandhi SI (2019) Video stitching using interacting multiple model based feature tracking. Multimed Tools Appl 78(2):1375–1397
Li S, Yuan L, Sun J et al (2015)Dual-feature warping-based motion model estimation. In: ICCV, pp 4283–4291
Li J, Wang Z, Lai S, Zhai Y, Zhang M (2018)Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans Multimedia 20(7):1672–1687
Lin WY, Liu S, Matsushita Y et al (2011) Smoothly varying affine stitching. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 345–352
Lin W, Cheng M, Lu J et al (2014) Bilateral functions for global motion modeling. In: Comput Vis ECCV, pp 341–356
Lin CC, Pankanti SU, Ramamurthy KN, Aravkin AY (2015) Adaptive as-natural-as-possible image stitching. In: Proc IEEE Conf. Comput Vis Pattern Recogn, pp 1155–1163
Lin K, Jiang N, Cheong LF et al (2016) Seagull: seam-guided local alignment for parallax-tolerant image stitching. In: Proc Eur Conf Comput Vis, pp 370–385
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Nie L, Lin C, Liao K, Liu M, Zhao Y (2020) A view-free image stitching network based on global homography. J Vis Commun Image Represent 73:102950
Qiu Z, Tang H, Tian D (2009)Non-rigid medical image registration based on the thin-plate spline algorithm. Proc WRI World Congr Comput Sci Inf Eng 2:522–527
Rublee E, Rabaud V, Konolige K, Bradski G (2011) Orb: an efficient alternative to sift or surf. In: Proc of ICCV, pp 2564–2571
Sawhney HS, Kumar R (1999) True multi-image alignment and its application to mosaicing and lens distortion correction. IEEE Trans Pattern Anal Mach Intell 21:235–243
Wu G, Lin Z, Ding G, Ni Q, Han J (2020) On aggregation of unsupervised deep binary descriptor with weak bits. IEEE Trans Image Process 29:9266–9278
Yan W, Yue G, Fang Y, Chen H, Tang C, Jiang G (2020) Perceptual objective quality assessment of stereoscopic stitched images. Signal Process 172:107541
Zaragoza J, Chin TJ, Tran QH et al (2014)As-projective-as-possible image stitching with moving dlt. IEEE Trans Pattern Anal Mach Intell 36(7):1285–1298
Zhang G, He Y, Chen W, Jia J, Bao H (2016)Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans Image Process 25(7):3099–3111
Zhao S, Lau T, Luo J et al (2019) Unsupervised 3D end-to-end medical image registration with volume tweening network. IEEE J Biomed Health Inform 24(5):1394–1404
Zhou H, Kuang Y, Yu Z, Ren S, Zhang Y, Lu T, Ma J (2018) Image deformation with vector-field interpolation based on MRLS-TPS. IEEE Access 6:75886–75898
Acknowledgments
This paper was supported by the National Natural Science Foundation of China (No. 62071326) and the National Natural Science Foundation of China (No. 61674115).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shi, Z., Wang, P., Cao, Q. et al. Misalignment-eliminated warping image stitching method with grid-based motion statistics matching. Multimed Tools Appl 81, 10723–10742 (2022). https://doi.org/10.1007/s11042-022-12064-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-12064-2