Skip to main content
SpringerLink
Log in

Fast image blending for high-quality panoramic images on mobile phones

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a fast image blending approach for combining a set of registered images into a composite mosaic with no visible seams and minimal texture distortion on mobile phones. A unique seam image is generated using two-pass nearest distance transform, which is independent on the order of input images and has good scalability. Each individual mask can be extracted from this seam image quickly. To promote blending speed and reduce memory usage in building high resolution image mosaics on mobile phones, the seam image and mask images are compressed using run-length encoding, and all the following mask operations are built on run-length encoding scheme. Moreover, single instruction multiple data instruction set is used in Gaussian and Laplacian pyramids construction to improve the blending speed further. The use of run-length encoding for masks processing leads to reduced memory requirements and a compact storage of the mask data, and the use of single instruction multiple data instruction set achieves better parallelism and faster execution speed on mobile phones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Agarwala A (2007) Efficient gradient-domain compositing using quadtrees. ACM Trans Graphics (TOG) 26(3):94

    Article  Google Scholar 

  2. Agarwala A, Dontcheva M, Agrawala M, Drucker S, Colburn A, Curless B, Salesin D, Cohen M (2004) Interactive digital photomontage. ACM Trans Graphics (TOG) 23(3):294–302

    Article  Google Scholar 

  3. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26 (3):10–19

    Article  Google Scholar 

  4. Bie X, Wang W, Sun H, Huang H, Zhang M (2013) Intent-aware image cloning. Vis Comput 29(6-8):599–608

    Article  Google Scholar 

  5. Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73

    Article  Google Scholar 

  6. Burt PJ, Adelson EH (1983) A multiresolution spline with application to image mosaics. ACM Trans Graphics (TOG) 2(4):217–236

    Article  Google Scholar 

  7. Chang C, Chou C, Chang EY (2017) Clkn: Cascaded lucas-kanade networks for image alignment. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 3777–3785

  8. DeTone D, Malisiewicz T, Rabinovich A (2016) Deep image homography estimation. arXiv:abs/1606.03798

  9. Ding M, Tong RF (2010) Content-aware copying and pasting in images. Vis Comput 26(6-8):721–729

    Article  Google Scholar 

  10. Efros A, Freeman W (2001) Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, SIGGRAPH ’01. ACM, New York, pp 341–346

  11. Farbman Z, Hoffer G, Lipman Y, Cohen-Or D, Lischinski D (2009) Coordinates for instant image cloning. ACM Trans Graph 28(3):67:1–67:9

    Article  Google Scholar 

  12. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems 27. Curran Associates Inc, pp 2672–2680

  13. Gracias N, Mahoor M, Negahdaripour S, Gleason A (2009) Fast image blending using watersheds and graph cuts. Image Vis Comput 27 (5):597–607

    Article  Google Scholar 

  14. Hao C, Chen Y, Wu W, Wu E (2015) An iterated randomized search algorithm for large-scale texture synthesis and manipulations. Vis Comput 31(11):1447–1458

    Article  Google Scholar 

  15. He K, Chang H, Sun J (2013) Rectangling panoramic images via warping. ACM Trans Graphics (TOG) 32(4):1–10

    MATH  Google Scholar 

  16. Jia J, Sun J, Tang CK, Shum HY (2006) Drag-and-drop pasting. ACM Trans Graph 25(3):631–637

    Article  Google Scholar 

  17. Kolmogorov V, Zabin R (2004) What energy functions can be minimized via graph cuts?. IEEE Trans Patt Anal and Mach Intel 26(2):147–159

    Article  Google Scholar 

  18. Kwatra V, Schödl A, Essa I, Turk G, Bobick A (2003) Graphcut textures: Image and video synthesis using graph cuts. In: ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03. ACM, New York, pp 277–286

  19. Levin A, Zomet A, Peleg S, Weiss Y (2004) Seamless image stitching in the gradient domain. In: Proceedings of the European conference on computer vision (ECCV ’04). Springer, Prague, pp 377–389

  20. Liu G, Reda FA, Shih KJ, Wang TC, Tao A, Catanzaro B (2018) Image inpainting for irregular holes using partial convolutions. In: The european conference on computer vision (ECCV)

  21. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J of Comput Vis 60(2):91–110

    Article  Google Scholar 

  22. Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Patt Anal and Mach Intel 36(11):2227–2240

    Article  Google Scholar 

  23. Pérez P, Gangnet M, Blake A (2003) Poisson image editing. ACM Trans Graphics (TOG) 22(3):313–318

    Article  Google Scholar 

  24. Rocco I, Arandjelovic R, Sivic J (2017) Convolutional neural network architecture for geometric matching. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 39–48

  25. Summa B, Tierny J, Pascucci V (2012) Panorama weaving: fast and flexible seam processing. ACM Trans Graph 31(4):83:1–83:11

    Article  Google Scholar 

  26. Szeliski R, Uyttendaele M, Steedly D (2011) Fast poisson blending using multi-splines. In: Proceedings of the 2011 IEEE hy (ICCP). IEEE, Pittsburgh, pp 1–8

  27. Wu H, Zheng S, Zhang J, Huang K (2019) GP-GAN: Towards Realistic high-resolution image blending. In: Proceedings of the 27th ACM international conference on multimedia. Association for Computing Machinery, New York, pp 2487–2495

  28. Xie ZF, Shen Y, Ma LZ, Chen ZH (2010) Seamless video composition using optimized mean-value cloning. Vis Comput 26(6-8):1123–1134

    Article  Google Scholar 

  29. Xiong Y, Pulli K (2010) Fast panorama stitching for high-quality panoramic images on mobile phones. IEEE Trans Consum Electron 56(2):298–306

    Article  Google Scholar 

  30. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR)

Download references

Acknowledgements

This research was funded by the Natural Science Foundation of China under Grant No. 61662072.

Author information

Authors and Affiliations

  1. School of Big Data and Intelligent Engineering, Southwest Forestry University, Kunming, Yunnan, 650224, China

    Yili Zhao

Authors
  1. Yili Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yili Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y. Fast image blending for high-quality panoramic images on mobile phones. Multimed Tools Appl 80, 499–516 (2021). https://doi.org/10.1007/s11042-020-09717-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09717-5

Keywords

Search

Navigation