Skip to main content
Log in

Image stitching algorithm based on semantics-preserving warps

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Generating naturally stitched images is a challenging task due to the presence of the parallax in images. In this paper, we propose a semantic-based method to handle the distortions and artifacts generated in the process of stitching. Firstly, a semantic-based random sample consensus algorithm is proposed to obtain the corresponding feature points, which can effectively eliminate mismatch points. The obtained corresponding features are used to align the image and establish the global structure information. Then, we propose semantics-preserving warps based on mesh optimization. The global structure information, semantic information and the corresponding features are combined to warp the images. Experimental results show that our algorithm can provide accurate alignment results while effectively reducing image distortions in both overlapping and non-overlapping regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availibility statement

This article uses a public dataset: TuSimnpleLane Detection Challenge Dataset.

References

  1. Wang, L., Yu, W., Li, B.: Multi-scenes image stitching based on autonomous driving. In: 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) 1, pp. 694–698 (2020)

  2. Zhang, J., Liu, T., Yin, X., Wang, X., Zhang, K., Xu, J., Wang, D.: An improved parking space recognition algorithm based on panoramic vision. Multimedia Tools Appl. 80, 18181–18209 (2021)

    Article  MATH  Google Scholar 

  3. Luo, X., Li, Y., Yan, J., Guan, X.: Image stitching with positional relationship constraints of feature points and lines. Pattern Recognit. Lett. 135, 431–440 (2020)

    Article  MATH  Google Scholar 

  4. Zhang, J., Yin, X., Luan, J., Liu, T.: An improved vehicle panoramic image generation algorithm. Multimedia Tools Appl. 78, 27663–27682 (2019)

    Article  MATH  Google Scholar 

  5. Zhang, J., Pan, D., Zhang, K., Jin, J.J., Ma, Y.-Y., Chen, M.: Underwater single-image restoration based on modified generative adversarial net. SIViP 17, 1153–1160 (2022)

    Article  MATH  Google Scholar 

  6. Brown, M.A., Lowe, D.G.: Recognising panoramas. In: Proceedings Ninth IEEE International Conference on Computer Vision, pp. 1218–12252 (2003)

  7. Brown, M.A., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)

    Article  Google Scholar 

  8. Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. CVPR 2011, 49–56 (2011)

    Google Scholar 

  9. Zaragoza, J., Chin, T.-J., Tran, Q.-H., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving dlt. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2339–2346 (2013)

  10. Jia, Q., Li, Z., Fan, X., Zhao, H., Teng, S., Ye, X., Latecki, L.J.: Leveraging line-point consistence to preserve structures for wide parallax image stitching. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12181–12190 (2021)

  11. Li, N., Xu, Y., Wang, C.: Quasi-homography warps in image stitching. IEEE Trans. Multimedia 20, 1365–1375 (2017)

    Article  MATH  Google Scholar 

  12. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269 (2014)

  14. Lin, C.-C., Pankanti, S., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1155–1163 (2015)

  15. Chang, C.-H., Sato, Y., Chuang, Y.-Y.: Shape-preserving half-projective warps for image stitching. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)

  16. Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: European Conference on Computer Vision (2016)

  17. Liao, T., Li, N.: Single-perspective warps in natural image stitching. IEEE Trans. Image Process. 29, 724–735 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  18. Du, P., Ning, J., Cui, J., Huang, S., Wang, X., Wang, J.: Geometric structure preserving warp for natural image stitching. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3678–3686 (2022)

  19. Zhang, J., Xiu, Y.: Image stitching based on human visual system and sift algorithm. Visual Comput. 1–13 (2023)

  20. Lin, K., Jiang, N., Cheong, L.F., Do, M.N., Lu, J.: Seagull: Seam-guided local alignment for parallax-tolerant image stitching. In: European Conference on Computer Vision (2016)

  21. Li, A., Guo, J., Guo, Y.: Image stitching based on semantic planar region consensus. IEEE Trans. Image Process. 30, 5545–5558 (2020)

    Article  MATH  Google Scholar 

  22. Zheng, J., Wang, Y., Wang, H., Li, B., Hu, H.-M.: A novel projective-consistent plane based image stitching method. IEEE Trans. Multimedia 21, 2561–2575 (2019)

    Article  MATH  Google Scholar 

  23. Zhang, Y., Lai, Y.-K., Zhang, F.-L.: Content-preserving image stitching with piecewise rectangular boundary constraints. IEEE Trans. Visual Comput. Graph. 27(7), 3198–3212 (2021). https://doi.org/10.1109/TVCG.2020.2965097

    Article  MATH  Google Scholar 

  24. Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Deep rectangling for image stitching: a learning baseline. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5730–5738 (2022). https://doi.org/10.1109/CVPR52688.2022.00565

  25. DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation. CoRR abs/1606.03798 (2016) arXiv:1606.03798

  26. Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model

  27. Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Unsupervised deep image stitching: Reconstructing stitched features to images. IEEE Trans. Image Process. 30, 6184–6197 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Parallax–Tolerant Unsupervised Deep Image Stitching (2023)

  29. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Semantic understanding of scenes through the ade20k dataset. Int. J. Comput. Vis. 127, 302–321 (2016)

  30. Zhang, Z.: Parameter estimation techniques: a tutorial with application to conic fitting. Image Vis. Comput. 15, 59–76 (1997)

    Article  MATH  Google Scholar 

  31. Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3d video stabilization. ACM Trans. Graph. 28, 44 (2009)

    Article  Google Scholar 

  32. Liu, S., Yuan, L., Tan, P., Sun, J.: Bundled camera paths for video stabilization. ACM Trans. Graph. (TOG) 32, 1–10 (2013)

Download references

Funding

(1) National Key Research and Development Program of China (2017YFB0102500), (2) National Natural Science Foundation of China (62172186).

Author information

Authors and Affiliations

Authors

Contributions

(1) Jindong Zhang is responsible for providing experimental platform and paper review. (2) Ninan Jiang is responsible for writing the paper and completing the experiment.

Corresponding author

Correspondence to Jindong Zhang.

Ethics declarations

Conflict of interest

I declare that the authors have no Conflict of interest as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (csv 5 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Jiang, N. Image stitching algorithm based on semantics-preserving warps. SIViP 19, 40 (2025). https://doi.org/10.1007/s11760-024-03585-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03585-4

Keywords

Navigation