Skip to main content
SpringerLink
Log in

A simple yet effective image stitching with computational suture zone

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Image stitching is the process of combining two or more photographic images with spatially overlapping areas into a wider-view panorama accommodating the full-scale information. It suffers from ghosting or obvious fracture sometimes after stitching in the overlapped areas, especially when moving targets or foreground targets occurs in blending areas. For the purpose of eliminating the stitching trace and avoiding the ghosting caused by foreground targets in overlapped areas, in this work, a novel image stitching technique by a computational blending zone is proposed. In specific, a dynamic programming of optimal seam-line selection is proposed by exploiting the minimization of a defined energy function based on color, gradient and similarity within the overlapped regions. Based on the optimal seam-line obtained, an optimal region fixed in terms of a proposed gray characteristic function, which expanded from the selected suture line to both sides, is provided for image blending to acquire the final panoramic image. The reference image and the target image are stitched into a panoramic image according to the selected optimal seam-line and suitable blending region. Some experiments are conducted to show the effectiveness of the proposed technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Wang, Z., Yang, Z.: Review on image-stitching techniques. Multimedia Syst. 26(4), 413–430 (2020)

    Article  Google Scholar 

  2. Ren, W., Pan, J., Zhang, H., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks with holistic edges. Int. J. Comput. Vis. 128(1), 240–259 (2020)

    Article  Google Scholar 

  3. Liu, R., Ma, L., Zhang, J., Fan, X., Luo, Z.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 10561–10570 (2021)

  4. Yuan, Y., Fang, F., Zhang, G.: Superpixel-based seamless image stitching for UAV images. IEEE Trans. Geosci. Remote Sens. 59(2), 1565–1576 (2021)

    Article  Google Scholar 

  5. Haskins, G., Kruger, U., Yan, P.: Deep learning in medical image registration: a survey. Mach. Vis. Appl. 31(1), 1–18 (2020)

    Google Scholar 

  6. Adel, E., Elmogy, M., Elbakry, H.: Image stitching based on feature extraction techniques: a survey. Int. J. Comput. Appl. 99(6), 1–8 (2014)

    Google Scholar 

  7. Öfverstedt, J., Lindblad, J., Sladoje, N.: Fast and robust symmetric image registration based on distances combining intensity and spatial information. IEEE Trans. Image Process. 28(7), 3584–3597 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  8. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  9. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  10. Alcantarilla, P.F., Bartoli, A., Davison, A.J.: Kaze features. In: European Conference on Computer Vision, pp. 214–227 (2012)

  11. Alcantarilla, P.F., Solutions, T.: Fast explicit diffusion for accelerated features in nonlinear scale spaces. In: British Machine Vision Conference, pp. 214–227 (2013)

  12. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to SIFT or SURF. In: International Conference on Computer Vision, pp. 2564–2571 (2011)

  13. Tareen, S.A.K., Saleem, Z.: A comparative analysis of sift, surf, kaze, akaze, orb, and brisk. In: International Conference on Computing, Mathematics and Engineering Technologies, pp. 10–17 (2018)

  14. Wang, G., Zhai, Z., Xu, B., Cheng, Y.: A parallel method for aerial image stitching using orb feature points. In: International Conference on Computer and Information Science, pp. 548–555 (2017)

  15. Qu, Z., Li, J., Bao, K.-H., Si, Z.-C.: An unordered image stitching method based on binary tree and estimated overlapping area. IEEE Trans. Image Process. 29(1), 6734–6744 (2020)

    Article  MATH  Google Scholar 

  16. Zamir, A.R., Shah, M.: Image geo-localization based on multiple nearest neighbor feature matching using generalized graphs. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1546–1558 (2014)

    Article  Google Scholar 

  17. Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. Int. Confer. Comput. Vis. Theory Appl. 2(2), 331–340 (2009)

    Google Scholar 

  18. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  19. Liu, R., Mu, P., Yuan, X., Zeng, S., Zhang, J.: A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. In: International Conference on Machine Learning, pp. 6305–6315. PMLR (2020)

  20. Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.-H.: Gated fusion network for single image dehazing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3253–3261 (2018)

  21. Peng, X., Feng, J., Xiao, S., Yau, W.-Y., Zhou, J.T., Yang, S.: Structured autoencoders for subspace clustering. IEEE Trans. Image Process. 27(10), 5076–5086 (2018)

    Article  MathSciNet  Google Scholar 

  22. Peng, X., Zhu, H., Feng, J., Shen, C., Zhang, H., Zhou, J.T.: Deep clustering with sample-assignment invariance prior. IEEE Trans. Neural Netw Learn Syst 31(11), 4857–4868 (2019)

    Article  MathSciNet  Google Scholar 

  23. Sun, D., Yang, X., Liu, M.-Y., Kautz, J.: Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In: Computer Vision and Pattern Recognition, pp. 8934–8943 (2018)

  24. Tian, L., Tu, Z., Zhang, D., Liu, J., Li, B., Yuan, J.: Unsupervised learning of optical flow with CNN-based non-local filtering. IEEE Trans. Image Process. 29(1), 8429–8442 (2020)

    Article  MATH  Google Scholar 

  25. Liao, K., Lin, C., Zhao, Y., Xu, M.: Model-free distortion rectification framework bridged by distortion distribution map. IEEE Trans. Image Process. 29(3), 3707–3718 (2020)

    Article  MATH  Google Scholar 

  26. Li, H., Wu, X.-J.: Densefuse: a fusion approach to infrared and visible images. IEEE Trans. Image Process. 28(5), 2614–2623 (2019)

    Article  MathSciNet  Google Scholar 

  27. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  28. Zaragoza, J., Chin, T.-J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: Computer Vision and Pattern Recognition, pp. 2339–2346 (2013)

  29. Chang, C.-H., Sato, Y., Chuang, Y.-Y.: Shape-preserving half-projective warps for image stitching. In: Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)

  30. Lin, C.-C., Pankanti, S.U., Natesan Ramamurthy, K., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: Computer Vision and Pattern Recognition, pp. 1155–1163 (2015)

  31. Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: European Conference on Computer Vision, pp. 186–201 (2016)

  32. Li, L., Yao, J., Lu, X., Tu, J., Shan, J.: Optimal seamline detection for multiple image mosaicking via graph cuts. ISPRS J. Photo Gram. Remote Sens. 113(6), 1–16 (2016)

    Google Scholar 

  33. Lee, D., Lee, S.: Seamless image stitching by homography refinement and structure deformation using optimal seam pair detection. J. Electron. Imaging 26(6), 63–66 (2017)

    Article  Google Scholar 

  34. Kerschner, M.: Seamline detection in colour orthoimage mosaicking by use of twin snakes. ISPRS J. Photogram. Remote. Sens. 56(1), 53–64 (2001)

    Article  Google Scholar 

  35. Pan, J., Wang, M., Ma, D., Zhou, Q., Li, J.: Seamline network refinement based on area voronoi diagrams with overlap. IEEE Trans. Geosci. Remote Sens. 52(3), 1658–1666 (2014)

    Article  Google Scholar 

  36. Zhao, G., Lin, L., Tang, Y.: A new optimal seam finding method based on tensor analysis for automatic panorama construction. Pattern Recogn. Lett. 34(3), 308–314 (2013)

    Article  Google Scholar 

  37. Kaur, H., Koundal, D., Kadyan, V.: Image fusion techniques: a survey. Arch. Comput. Methods Eng. 28(7), 4425–4447 (2021)

    Article  Google Scholar 

  38. Li, X., Hui, N., Shen, H., Fu, Y., Zhang, L.: A robust mosaicking procedure for high spatial resolution remote sensing images. ISPRS J. Photogram. Remote Sens. 109(11), 108–125 (2015)

    Article  Google Scholar 

  39. Herrmann, C., Wang, C., Bowen, R.S., Keyder, E., Krainin, M., Liu, C., Zabih, R.: Robust image stitching with multiple registrations. In: European Conference on Computer Vision, pp. 53–67 (2018)

  40. Li, L., Tu, J., Gong, Y., Yao, J., Li, J.: Seamline network generation based on foreground segmentation for orthoimage mosaicking. ISPRS J. Photogram. Remote Sens. 148, 41–53 (2019)

    Article  Google Scholar 

  41. Gao, J., Li, Y., Chin, T.-J., Brown, M.S.: Seam-driven image stitching. In: Eurographics (Short Papers), pp. 45–48 (2013)

  42. Lin, K., Jiang, N., Cheong, L.F., Do, M., Lu, J.: Seagull: Seam-guided local alignment for parallax-tolerant image stitching. In: European Conference on Computer Vision, pp. 370–385 (2016)

  43. Zhou, D.-F., He, M.-Y., Yang, Q.: A robust seamless image stitching algorithm based on feature points. Meas. Control Technol. 28(6), 32–36 (2009)

    Google Scholar 

  44. Li, H., Luo, J., Huang, C., Yang, Y., Xie, S.: An adaptive image-stitching algorithm for an underwater monitoring system. Int. J. Adv. Rob. Syst. 11(10), 166–176 (2014)

    Article  Google Scholar 

  45. Popovic, V., Leblebici, Y.: Fir filters for hardware-based real-time multi-band image blending. In: Real-Time Image and Video Processing, vol. 94, pp. 940–947 (2015)

  46. Chen, M., Nian, R., He, B., Qiu, S., Liu, X., Yan, T.: Underwater image stitching based on sift and wavelet fusion. In: OCEANS 2015-Genova, pp. 1–4 (2015)

  47. Srivastava, R., Prakash, O., Khare, A.: Local energy-based multimodal medical image fusion in curvelet domain. IET Comput. Vis. 10(6), 513–527 (2016)

    Article  Google Scholar 

  48. Yang, H., Long, Y., Lin, J., Zhang, F., Chen, Z.: A seismic interpolation and denoising method with curvelet transform matching filter. Acta Geophys. 65(5), 1029–1042 (2017)

    Article  Google Scholar 

  49. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 62002160, 62072245, 62072238, and 61703201), the Natural Science Foundation of Jiangsu Province (Grant Nos. BK20211520 and BK20201042), and the Science Foundation of Nanjing Institute of Technology (Grant No. ZKJ202003).

Author information

Author notes

  1. Yang Gao, Yi Xu, Yunbin Huang, Yanming Yu and Xiangbo Shu are contributed equally to this work.

Authors and Affiliations

  1. Artificial Intelligence Industrial Technology Research Institute, Nanjing Institute of Technology, Nanjing, China

    Jiachao Zhang, Yang Gao & Yi Xu

  2. School of Electronics and Communication Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China

    Yunbin Huang

  3. Guohua (Hami) New Energy Co., Ltd., Hami, China

    Yanming Yu

  4. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Xiangbo Shu

Authors
  1. Jiachao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunbin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanming Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiangbo Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiachao Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Gao, Y., Xu, Y. et al. A simple yet effective image stitching with computational suture zone. Vis Comput 39, 4915–4928 (2023). https://doi.org/10.1007/s00371-022-02637-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02637-5

Keywords

Search

Navigation