Skip to main content
SpringerLink
Log in

Intent-aware image cloning

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Currently, gradient domain methods are popular for producing seamless cloning of a source image patch into a target image. However, structure conflicts between the source image patch and the target image may generate artifacts, preventing the general practices. In this paper, we tackle the challenge by incorporating the users’ intent in outlining the source patch, where the boundary drawn generally has different appearances from the objects of interest. We first reveal that artifacts exist in the over-included region, the region outside the objects of interest in the source patch. Then we use the diversity from the boundary to approximately distinguish the objects from the over-included region, and design a new algorithm to make the target image adaptively take effects in blending. So the structure conflicts can be efficiently suppressed to remove the artifacts around the objects of interest in the composite result. Moreover, we develop an interpolation measure to composite the final image rather than solving a Poisson equation, and speed up the interpolation by treating pixels in clusters and using hierarchical sampling techniques. Our method is simple to use for instant and high-quality image cloning, in which users only need to outline a region of interested objects to process. Our experimental results have demonstrated the effectiveness of our cloning method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agarwala, A.: Efficient gradient-domain compositing using quadtrees. In: SIGGRAPH ’07: ACM SIGGRAPH, p. 94. ACM, New York (2007)

    Google Scholar 

  2. An, X., Pellacini, F.: Appprop: all-pairs appearance space edit propagation. In: SIGGRAPH ’08: ACM SIGGRAPH, pp. 1–9. ACM, New York, NY, USA (2008)

    Google Scholar 

  3. Chen, T., Cheng, M.M., Tan, P., Shamir, A., Hu, S.M.: Sketch2photo: Internet image montage. ACM Trans. Graph. 28(5), 124:1–124:10 (2009)

    Google Scholar 

  4. Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 82:1–82:10 (2012) (Proceedings of SIGGRAPH 2012)

    Article  Google Scholar 

  5. Ding, M., Tong, R.F.: Content-aware copying and pasting in images. Vis. Comput. 26(6–8), 721–729 (2010)

    Article  Google Scholar 

  6. Du, H., Jin, X.: Object cloning using constrained mean value interpolation. Vis. Comput. 29(3), 217–229 (2013)

    Article  Google Scholar 

  7. Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Lischinski, D.: Coordinates for instant image cloning. ACM Trans. Graph. 28(3), 1–9 (2009)

    Article  Google Scholar 

  8. Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. Comput. Graph. Forum 29(2), 575–584 (2010)

    Article  Google Scholar 

  9. Jeschke, S., Cline, D., Wonka, P.: A GPU Laplacian solver for diffusion curves and Poisson image editing. In: SIGGRAPH Asia ’09: ACM SIGGRAPH Asia, pp. 1–8. ACM, New York (2009)

    Chapter  Google Scholar 

  10. Jia, J., Sun, J., Tang, C.K., Shum, H.Y.: Drag-and-drop pasting. In: SIGGRAPH ’06: ACM SIGGRAPH, pp. 631–637. ACM, New York (2006)

    Google Scholar 

  11. Kazhdan, M., Hoppe, H.: Streaming multigrid for gradient-domain operations on large images. ACM Trans. Graph. 27(3), 1–10 (2008)

    Article  Google Scholar 

  12. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

    Article  Google Scholar 

  13. Levin, A., Rav-Acha, A., Lischinski, D.: Spectral matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1699–1712 (2008)

    Article  Google Scholar 

  14. McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. In: SIGGRAPH ’05: ACM SIGGRAPH, pp. 567–576. ACM, New York (2005)

    Google Scholar 

  15. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)

    Article  Google Scholar 

  16. Smith, A.R., Blinn, J.F.: Blue screen matting. In: SIGGRAPH ’96: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 259–268. ACM, New York (1996)

    Chapter  Google Scholar 

  17. Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. In: SIGGRAPH ’04: ACM SIGGRAPH, pp. 315–321. ACM, New York (2004)

    Google Scholar 

  18. Sunkavalli, K., Johnson, M.K., Matusik, W., Pfister, H.: Multi-scale image harmonization. ACM Trans. Graph. 29(4), 125:1–125:10 (2010) (Proc. ACM SIGGRAPH)

    Article  Google Scholar 

  19. Szeliski, R.: Locally adapted hierarchical basis preconditioning. In: SIGGRAPH ’06: ACM SIGGRAPH, pp. 1135–1143. ACM, New York (2006)

    Google Scholar 

  20. Wang, J., Cohen, M.F.: Simultaneous matting and compositing. In: ACM SIGGRAPH 2006 Sketches, SIGGRAPH ’06. ACM, New York (2006)

    Google Scholar 

  21. Xie, Z.F., Shen, Y., Ma, L.Z., Chen, Z.H.: Seamless video composition using optimized mean-value cloning. Vis. Comput. 26(6–8), 1123–1134 (2010)

    Article  Google Scholar 

  22. Zhang, Y., Tong, R.: Environment-sensitive cloning in images. Vis. Comput. 27(6–8), 739–748 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The work is supported by the Knowledge Innovation Program of the Chinese Academy of Sciences, the National Social Science Foundation of China (Project No. 12AZD118), RGC research grants (ref. 416311, 416212) and UGC direct grant for research (no. 2050485).

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, China

    Xiaohui Bie, Wencheng Wang & Minying Zhang

  2. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, China

    Hanqiu Sun

  3. Google Inc., Mountain View, CA, 94043, USA

    Haoda Huang

  4. University of Chinese Academy of Sciences, Beijing, China

    Xiaohui Bie & Minying Zhang

Authors
  1. Xiaohui Bie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wencheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanqiu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haoda Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Minying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiaohui Bie or Wencheng Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bie, X., Wang, W., Sun, H. et al. Intent-aware image cloning. Vis Comput 29, 599–608 (2013). https://doi.org/10.1007/s00371-013-0826-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-013-0826-0

Keywords

Search

Navigation