Skip to main content
SpringerLink
Log in

Object cloning using constrained mean value interpolation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present an object-cloning algorithm using constrained mean value interpolation over images. Different from previous methods, we first define the guidance vector field as a weighted gradient of the source and destination images in the gradient domain, which produces a similar Laplace equation to that of the original Poisson method but with different boundary conditions. Then, we use mean value coordinates to solve the new Laplace equation approximately instead of solving a large linear system. The cloned result matches well with the destination image on the luma and preserves the chroma of the source object as much as possible. Our cloned results are visually pleasing without smudging and discoloration artifacts. We also extend our approach to video object cloning. Experimental results demonstrate the effectiveness of our object cloning algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Agarwala, A.: Efficient gradient-domain compositing using quadtrees. ACM Trans. Graph. 26(3) (2007)

  2. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. In: Proceedings of IEEE CVPR (2006)

    Google Scholar 

  3. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

    Article  Google Scholar 

  4. Rhemann, C., Rother, C., Kohli, P., Gelautz, M.: A spatially varying PSF-based prior for alpha matting. In: Proceedings of IEEE CVPR (2010)

    Google Scholar 

  5. C. Corporation: Knockout User Guide (2002)

    Google Scholar 

  6. Guo, D., Sim, T.: Color me right-seamless image compositing. In: 13th Intl. Conf. on Computer Analysis of Images and Patterns (CAIP) (2009)

    Google Scholar 

  7. Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. Comput. Graph. Forum 29(2), 575–584 (2010)

    Article  Google Scholar 

  8. Floater, M.S., Kós, G., Reimers, M.: Mean value coordinates in 3D. Comput. Aided Geom. Des. 22(7), 623–631 (2005)

    Article  MATH  Google Scholar 

  9. Wang, H.-C., Ramesh, R., Narendra, A.: Seamless video editing. In: ICPR’04: Proceedings of the Pattern Recognition, 17th International Conference on (ICPR’04), vol. 3, pp. 858–861 (2004)

    Google Scholar 

  10. Bolz, J., Farmer, I., Grinspun, E., Schröder, P.: Sparse matrix solvers on the GPU: conjugate gradients and multigrid. ACM Trans. Graph. 22, 917–924 (2003)

    Article  Google Scholar 

  11. Jia, J., Sun, J., Tang, C.-K., Shum, H.-Y.: Drag-and-drop pasting. ACM Trans. Graph. 25(3), 631–637 (2006)

    Article  Google Scholar 

  12. Manson, J., Schaefer, S.: Moving least squares coordinates. Comput. Graph. Forum 29(5), 1517–1524 (2010)

    Article  Google Scholar 

  13. Mccann, J., Pollard, N.S.: Real-time gradient domain painting. ACM Trans. Graph. 27(3) (2008)

  14. Sun, J., Jia, J., Tang, C.-K., Shum, H.-Y.: Poisson matting. ACM Trans. Graph. 23(3) (2004)

  15. Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: Proceedings of IEEE CVPR, pp. 1–8 (2007)

    Google Scholar 

  16. Lalonde, J.-F., Hoiem, D., Efros, A.A., Rother, C., Winn, J., Criminisi, A.: Photo clip art. ACM Trans. Graph. 26(3), 3 (2007). (SIGGRAPH 2007)

    Article  Google Scholar 

  17. Hormann, K., Sukumar, N.: Maximum entropy coordinates for arbitrary polytopes. Comput. Graph. Forum 27(5), 1513–1520 (2008)

    Article  Google Scholar 

  18. Sunkavalli, K., Micah, J., Matusik, W., Pfister, H.: Multi-scale image harmonization. ACM Trans. Graph. 29(4) (2010)

  19. He, K.-M., Sun, J., Tang, X.: Fast matting using large kernel matting Laplacian matrices. In: Proceedings of IEEE CVPR, pp. 2165–2172 (2010)

    Google Scholar 

  20. Grady, L., Schiwietz, T., Aharon, S., Westermann, R.: Random walks for interactive alpha-matting. In: Proceedings of the Fifth IASTED International Conference on Visualization, Imaging and Image Processing, pp. 423–429 (2005)

    Google Scholar 

  21. Levin, A., Rav-Acha, A., Lischinski, D.: Spectral Matting (2007)

    Google Scholar 

  22. Bratkova, M., Boulos, S., Shirley, P.: oRGB: a practical opponent color space for computer graphics. IEEE Comput. Graph. Appl. 29(1), 42–55 (2009)

    Article  Google Scholar 

  23. Ding, M., Tong, R.-F.: Content-aware copying and pasting in images. Vis. Comput. 26(6–8), 721–729 (2010)

    Article  Google Scholar 

  24. Kazhdan, M., Hoppe, H.: Streaming multigrid for gradient-domain operations on large images. ACM Trans. Graph. 27(3) (2008)

  25. Tao, M.W., Johnson, M.K., Paris, S.: Error-tolerant image compositing. In: European Conference on Computer Vision (2010)

    Google Scholar 

  26. Floater, M.S.: Mean value coordinates. Comput. Aided Geom. Des. 20(1), 19–27 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  27. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)

    Article  Google Scholar 

  28. Szeliski, R.: Locally adapted hierarchical basis preconditioning. ACM Trans. Graph. 25, 1135–1143 (2006)

    Article  Google Scholar 

  29. Wang, R., Chen, W.-F., Pan, M.-H., Bao, H.-J.: Harmonic coordinates for real-time image cloning. J. Zhejiang Univ. Sci. 11, 690–698 (2010)

    Google Scholar 

  30. Ruzon, M.A., Tomasi, C.: Alpha estimation in natural images. In: Proceedings of IEEE CVPR, pp. 18–25 (2000)

    Google Scholar 

  31. Jeschke, S., Cline, D., Wonka, P.: A GPU Laplacian solver for diffusion curves and Poisson image editing. ACM Trans. Graph. 28(5) (2009)

  32. Lee, S.-Y., Lee, I.-K.: Improved coordinate-based image and video cloning algorithm. In: ACM SIGGRAPH Asia Posters (2009)

    Google Scholar 

  33. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., Hu, S.-M.: Sketch2photo:internet image montage. ACM Transactions on Graphics 28(5), 124:1–124:10 (2009)

    Google Scholar 

  34. Ju, T., Schaefer, S., Joe, W.: Mean value coordinates for closed triangular meshes. ACM Trans. Graph. 24(3), 561–566 (2005)

    Article  Google Scholar 

  35. Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: International Conference on Computer Vision (2007)

    Google Scholar 

  36. Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. ACM Trans. Graph. 28(3), 70:1–70:11 (2009)

    Article  Google Scholar 

  37. Guan, Y., Chen, W., Liang, X., Ding, Z., Peng, Q.-S.: Easy matting—a stroke based approach for continuous image matting. Comput. Graph. Forum 25(3), 567–576 (2006)

    Article  Google Scholar 

  38. Mishima, Y.: Soft edge chroma-key generation based upon hexoctahedral color space. US Patent 5,355,174 (1994)

  39. Zhang, Y., Tong, R.-F.: Environment-sensitive cloning in images. Vis. Comput. 27, 739–748 (2011)

    Article  Google Scholar 

  40. Zheng, Y.-J., Kambhamettu, C.: Learning based digital matting. In: International Conference on Computer Vision (2009)

    Google Scholar 

  41. Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of IEEE CVPR, vol. 2, pp. 264–271 (2001)

    Google Scholar 

  42. Farbman, Z., Hoffer, G., Lipman, Y., Cohen-Or, D., Lischinski, D.: Coordinates for instant image cloning. ACM Trans. Graph. 28(3) (2009)

  43. Xie, Z.-F., Shen, Y., Ma, L.-Z., Chen, Z.-H.: Seamless video composition using optimized mean-value cloning. Vis. Comput. 26(6–8), 1123–1134 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

Xiaogang Jin was supported by Zhejiang Provincial Natural Science Foundation of China (Grant No. Z1110154), NSFC-MSRA Joint Funding (Grant No. 60970159), and the National Natural Science Foundation of China (Grant No. 60933007). Hui Du was supported by Zhejiang Provincial Science and Technology Planning Project (Grant No. 2010C31090) and Scientific Research Fund of Zhejiang Provincial Education Department (Grant No. Y201016221).

Author information

Authors and Affiliations

  1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China

    Hui Du & Xiaogang Jin

  2. Zhejiang University of Media and Communications, Hangzhou, 310018, China

    Hui Du

Authors
  1. Hui Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaogang Jin.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(PDF 3.1 MB)

(AVI 8.8 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Du, H., Jin, X. Object cloning using constrained mean value interpolation. Vis Comput 29, 217–229 (2013). https://doi.org/10.1007/s00371-012-0722-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-012-0722-z

Keywords

Search

Navigation