Skip to main content
SpringerLink
Log in

Artistic stylization of face photos based on a single exemplar

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we propose a unified framework for fully automatic face photo stylization based on a single style exemplar. Constrained by the “single-exemplar” condition, where the numbers and varieties of patch samples are limited, we introduce flexibility in sample selection while preserving identity and content of the input photo. Based on the observation that many styles are characterized by unique color selections and texture patterns, we employ a two-phase procedure. The first phase searches a dense and semantic-aware correspondence between the input and the exemplar images, so that colors in the exemplar can be transferred to the input. The second phase conducts edge-preserving texture transfer, which preserves edges and contours of the input and mimics the textures of the exemplar at multiple scales. Experimental results demonstrate compelling visual effects and notable improvements over other state-of-the-art methods which are adapted for the same task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Burnt picture. http://bbs.wenxuecity.com/art/3047.html. Accessed 29 Feb 2016

  2. Rollip. http://www.rollip.com/start/?src=ws. Accessed 17 Feb 2016

  3. Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Computer Vision–ECCV 2010, pp. 29–43. Springer, New York (2010)

  4. Beier, T., Neely, S.: Feature-based image metamorphosis. In: ACM SIGGRAPH Computer Graphics, vol. 26, pp. 35–42. ACM, New York (1992)

  5. Choi, H.C., Sibbing, D., Kobbelt, L.: Nonparametric facial feature localization using segment-based eigenfeatures. Comput Intell Neurosci 501, 164290 (2015)

    Google Scholar 

  6. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.Q.: Color harmonization. In: ACM Transactions on Graphics (TOG), vol. 25, pp. 624–630. ACM, New York (2006)

  7. Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 82 (2012)

    Article  Google Scholar 

  8. Efros, A., Leung, T.K., et al.: Texture synthesis by non-parametric sampling. In: Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 1033–1038. IEEE (1999)

  9. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341–346. ACM, New York (2001)

  10. Gooch, B., Coombe, G., Shirley, P.: Artistic vision: painterly rendering using computer vision techniques. In: Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, pp. 83–ff. ACM, New York (2002)

  11. HaCohen, Y., Shechtman, E., Goldman, D.B., Lischinski, D.: Optimizing color consistency in photo collections. ACM Trans Graph 32(4), 38 (2013)

    Article  Google Scholar 

  12. Henstock, P.V., Chelberg, D.M.: Automatic gradient threshold determination for edge detection . IEEE Trans. Image Process 5(5), 784–787 (1996)

  13. Kagaya, M., Brendel, W., Deng, Q., Kesterson, T., Todorovic, S., Neill, P.J., Zhang, E.: Video painting with space-time-varying style parameters. Vis Comput Graph IEEE Trans 17(1), 74–87 (2011)

    Article  Google Scholar 

  14. Kaspar, A., Neubert, B., Lischinski, D., Pauly, M., Kopf, J.: Self tuning texture optimization. In: Computer Graphics Forum, vol. 34, pp. 349–359. Wiley Online Library, New York (2015)

  15. Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. In: ACM SIGGRAPH, pp. 277–286. ACM, New York (2003)

  16. Lewis, J.P.: Texture synthesis for digital painting. In: ACM SIGGRAPH Computer Graphics. vol. 18, pp. 245–252. ACM, New York (1984)

  17. Li, H., Liu, G., Ngan, K.N.: Guided face cartoon synthesis. Multimed. IEEE Trans. 13(6), 1230–1239 (2011)

    Article  Google Scholar 

  18. Li, H., Mould, D.: Contrast-aware halftoning. In: Computer Graphics Forum, vol. 29, pp. 273–280. Wiley Online Library, New York (2010)

  19. Li, H., Mould, D.: Artistic tessellations by growing curves. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, pp. 125–134. ACM, New York (2011)

  20. Li, H., Mould, D.: Content-sensitive screening in black and white. In: GRAPP, pp. 166–172 (2011)

  21. Li, H., Mould, D.: Structure-preserving stippling by priority-based error diffusion. In: Proceedings of Graphics Interface, pp. 127–134. Canadian Human-Computer Communications Society, Canada (2011)

  22. Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)

    Article  Google Scholar 

  23. Lu, C., Xu, L., Jia, J.: Combining sketch and tone for pencil drawing production. In: Proceedings of the Symposium on Non-Photorealistic Animation and Rendering, pp. 65–73. Eurographics Association, Manchester (2012)

  24. Neumann, L., Neumann, A.: Color style transfer techniques using hue, lightness and saturation histogram matching. In: Computational Aesthetics, pp. 111–122. Citeseer (2005)

  25. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM Transactions on Graphics (TOG), vol. 22, pp. 313–318. ACM, New York (2003)

  26. Shih, Y., Paris, S., Barnes, C., Freeman, W.T., Durand, F.: Style transfer for headshot portraits. ACM Trans Graph 33(4), 148 (2014)

    Article  Google Scholar 

  27. Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)

    Article  Google Scholar 

  28. Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)

    Article  Google Scholar 

  29. Zhang, W., Wang, X., Tang, X.: Lighting and pose robust face sketch synthesis. In: ECCV 2010, pp. 420–433. Springer, New York (2010)

  30. Zhang, Y., Dong, W., Deussen, O., Huang, F., Li, K., Hu, B.G.: Data-driven face cartoon stylization. ACM, New york (2014)

  31. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2879–2886. IEEE (2012)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Memorial University of Newfoundland, St. John’s, NL, A1B 3X5, Canada

    Zili Yi, Yang Li, Songyuan Ji & Minglun Gong

Authors
  1. Zili Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Songyuan Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Minglun Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minglun Gong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yi, Z., Li, Y., Ji, S. et al. Artistic stylization of face photos based on a single exemplar. Vis Comput 33, 1443–1452 (2017). https://doi.org/10.1007/s00371-016-1290-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1290-4

Keywords

Search

Navigation