Skip to main content
SpringerLink
Log in

Superpixel-based foreground-preserving image stitching

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image stitching aims to stitch multiple images with overlapping areas into a high-resolution image with natural appearance, no ghosting and no seam, so as to obtain a wide field of vision. And the stitching is required to be as fast as possible. In this article, we discuss the loss of foreground object caused by the seam-cut method for image stitching. Moreover, we propose an improved seam-cut method based on superpixel to solve this problem. The proposed method uses the matching information of feature points and superpixel segmentation to divide the scene into foreground and background. By adjusting the penalty value of foreground superpixels in the superpixel-based seam-cut method, the foreground object can avoid the cutting of optimal seam. The experimental results demonstrate that our method will not cause the ghosting of foreground object or the loss of foreground object in the stitching result in the scene with large parallax. In this method, the foreground object will be preserved completely.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  2. Chang, C.H, Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261

  3. Chen, Y.S., Chuang, Y.Y.: Natural image stitching with the global similarity prior. In: European conference on computer vision, pp. 186–201. Springer, (2016)

  4. Dai, Q., Fang, F., Li, J., Zhang, G., Zhou, A.: Edge-guided composition network for image stitching. Pattern Recogn. 118(9), 108019 (2021)

    Article  Google Scholar 

  5. DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation, arXiv preprint arXiv:1606.03798, (2016)

  6. Gao, J., Kim, S.J., Brown, M.S: Constructing image panoramas using dual-homography warping. In: CVPR 2011, pp. 49–56. IEEE, (2011)

  7. Gao, J., Li, Y., Chin, T-J., Brown, M.S.: Seam-driven image stitching. In: Eurographics (Short Papers), pp. 45–48, (2013)

  8. Hoang, V.D., Tran, D,P., Nhu N.G., Pham, V.H.: Deep feature extraction for panoramic image stitching. Springer, cham (2020)

  9. Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Gr.(tog) 22(3), 277–286 (2003)

    Article  Google Scholar 

  10. Le, H., Liu, F., Zhang, S., Agarwala, A.: Deep homography estimation for dynamic scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7652–7661, (2020)

  11. Li, J., Wang, Z., Lai, S., Zhai, Y., Zhang, M.: Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimed. 20(7), 1672–1687 (2017)

    Article  Google Scholar 

  12. Li, N., Liao, T., Wang, C.: Perception-based seam cutting for image stitching. SIViP 12(5), 967–974 (2018)

    Article  Google Scholar 

  13. Lin, C-C., Pankanti, S.U., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1163, (2015)

  14. Lin, K., Jiang, N., Cheong, L-F., Do, M., Lu, J.: Seagull: Seam-guided local alignment for parallax-tolerant image stitching. In European conference on computer vision, pp. 370–385. Springer, Cham (2016)

  15. Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., Cheong, L.F.: Smoothly varying affine stitching. In: CVPR 2011, pp. 345–352. IEEE, (2011)

  16. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  17. Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. IEEE Robot. Autom. Lett 3(3), 2346–2353 (2018)

    Article  Google Scholar 

  18. Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Unsupervised deep image stitching: reconstructing stitched features to images. IEEE Trans. Image Process. 30, 6184–6197 (2021)

    Article  Google Scholar 

  19. Nie, L., Lin, C., Liao, K., Liu, M., Zhao, Y.: A view-free image stitching network based on global homography. J. Vis. Commun. Image Represent. 73, 102950 (2020)

    Article  Google Scholar 

  20. Su, J., Cheng, H., Yang, L., Luo, A.: Robust spatial–temporal Bayesian view synthesis for video stitching with occlusion handling. Mach. Vis. Appl. 29(11), 1–14 (2017)

    Google Scholar 

  21. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  22. Zagrouba, E., Barhoumi, W., Amri, S.: An efficient image-mosaicing method based on multifeature matching. Mach. Vis. Appl. 20(3), 139–162 (2009)

    Article  Google Scholar 

  23. Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving dlt. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2339–2346, (2013)

  24. Zeng, L., Zhang, S., Zhang, J., Zhang, Y.: Dynamic image mosaic via sift and dynamic programming. Mach. Vis. Appl. 25(5), 1271–1282 (2014)

    Article  Google Scholar 

  25. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269, (2014)

  26. Zhang, H., Zhao, M.: Panoramic image stitching using double encoder-decoders. SN Comput. Sci. 2(2), 1–12 (2021)

    Article  Google Scholar 

  27. Zhang, J., Wang, C., Liu, S., Jia, L., Ye, N., Wang, J., Zhou, J., Sun, J.: Content-aware unsupervised deep homography estimation. In: European Conference on Computer Vision, pp. 653–669. Springer, (2020)

  28. Zhao, Q., Ma, Y., Zhu, C., Yao, C., Dai, F.: Image stitching via deep homography estimation. Neurocomputing 25(450), 219–229 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China [grant number 2016YFC0803000] and the National Natural Science Foundation of China [grant number 41371342].

Funding

This work was supported by the National Key Research and Development Program of China [grant number 2016YFC0803000] and the National Natural Science Foundation of China [grant number 41371342].

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, China

    Xinpeng Miao, Tao Qu & Xi Chen

  2. Electronic Information School, Wuhan University, Wuhan, China

    Chu He

Authors
  1. Xinpeng Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chu He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Qu.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, X., Qu, T., Chen, X. et al. Superpixel-based foreground-preserving image stitching. Machine Vision and Applications 34, 17 (2023). https://doi.org/10.1007/s00138-022-01363-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01363-1

Keywords

Search

Navigation