Skip to main content
SpringerLink
Log in

Augmenting photographs with textures using the Laplacian pyramid

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We introduce a method to stylize photographs with auxiliary textures, by means of the Laplacian pyramid. Laplacian pyramid coefficients from a synthetic texture are combined with the coefficients from the original image by means of a smooth maximum function. The final result is a stylized image which maintains the structural characteristics from the input, including edges, color, and existing texture, while enhancing the image with additional fine-scale details. Further, we extend patch-based texture synthesis to include a guidance channel so that texture structures are aligned with an orientation field, obtained through the image structure tensor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Akl, A., Yaacoub, C., Donias, M., Da Costa, J.P., Germain, C.: Texture synthesis using the structure tensor. IEEE Trans. Image Process. 24, 4082–4095 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  3. Ashikhmin, M.: Synthesizing natural textures. In: Proceedings of the 2001 Symposium on Interactive 3D Graphics, I3D ’01, pp. 217–226. ACM, New York, NY, USA (2001)

  4. Aubry, M., Paris, S., Hasinoff, S.W., Kautz, J., Durand, F.: Fast local Laplacian filters: theory and applications. ACM Trans. Graph. 33(5), 167:1–167:14 (2014)

    Article  Google Scholar 

  5. Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM Trans. Graph. 25(3), 637–645 (2006)

    Article  Google Scholar 

  6. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24 (2009)

    Article  Google Scholar 

  7. Brox, T., van den Boomgaard, R., Lauze, F., van de Weijer, J., Weickert, J., Mrázek, P., Kornprobst, P.: Adaptive structure tensors and their applications. In: J. Weickert, H. Hagen (eds.) Visualization and Processing of Tensor Fields. Mathematics and Visualization, chap. 2, pp. 17–47. Springer, Berlin (2006)

  8. Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

    Article  Google Scholar 

  9. Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)

    Article  Google Scholar 

  10. Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pp. 263–270. ACM, New York, NY, USA (1993)

  11. Cook, J.: Basic properties of the soft maximum. UT MD Anderson Cancer Center Department of Biostatistics Working Paper Series. Working Paper 70 (2011). http://biostats.bepress.com/cgi/viewcontent.cgi?article=1073&context=mdandersonbiostat

  12. Criminisi, A., Sharp, T., Rother, C., Pérez, P.: Geodesic image and video editing. ACM Trans. Graph. 29(5), 134:1–134:15 (2010)

    Article  Google Scholar 

  13. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  14. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of SIGGRAPH 98, SIGGRAPH ’01, pp. 341–346. ACM, New York, NY, USA (2001)

  15. Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proceedings of the International Conference on Computer Vision, ICCV ’99, vol. 2, p. 1033. IEEE Computer Society, Washington, DC, USA (1999)

  16. Elad, M., Milanfar, P.: Style transfer via texture synthesis. IEEE Trans. Image Process. 26(5), 2338–2351 (2017). https://doi.org/10.1109/TIP.2017.2678168

    Article  MathSciNet  MATH  Google Scholar 

  17. Fang, H., Hart, J.C.: Textureshop: texture synthesis as a photograph editing tool. ACM Trans. Graph. 23(3), 354–359 (2004)

    Article  Google Scholar 

  18. Fattal, R., Agrawala, M., Rusinkiewicz, S.: Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26(3), 51 (2007)

    Article  Google Scholar 

  19. Fišer, J., Jamriška, O., Simons, D., Shechtman, E., Lu, J., Asente, P., Lukáč, M., Sýkora, D.: Example-based synthesis of stylized facial animations. ACM Trans. Graph. 36(4), 155:1–155:11 (2017)

    Article  Google Scholar 

  20. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423 (2016)

  21. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, K.Q. Weinberger (eds.) Advances in Neural Information Processing Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014)

  22. Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’01, pp. 327–340. ACM, New York, NY, USA (2001)

  23. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: The IEEE International Conference on Computer Vision (ICCV) (2017)

  24. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II, pp. 694–711. Springer, Cham (2016)

    Chapter  Google Scholar 

  25. Khan, E.A., Reinhard, E., Fleming, R.W., Bülthoff, H.H.: Image-based material editing. ACM Trans. Graph. 25(3), 654–663 (2006)

    Article  Google Scholar 

  26. Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)

    Article  Google Scholar 

  27. Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)

    Article  Google Scholar 

  28. Lee, H., Seo, S., Yoon, K.: Extended papers from NPAR 2010: directional texture transfer with edge enhancement. Comput. Graph. 35(1), 81–91 (2011)

    Article  Google Scholar 

  29. Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)

    Article  Google Scholar 

  30. Okura, F., Vanhoey, K., Bousseau, A., Efros, A.A., Drettakis, G.: Unifying color and texture transfer for predictive appearance manipulation. Comput. Graph. Forum 34(4), 53–63 (2015)

    Article  Google Scholar 

  31. Paris, S., Hasinoff, S.W., Kautz, J.: Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans. Graph. 30(4), 68:1–68:12 (2011)

    Article  Google Scholar 

  32. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM Trans. Graph., vol. 22, pp. 313–318. ACM (2003)

  33. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)

    Article  Google Scholar 

  34. Sajjadi, M.S., Schölkopf, B., Hirsch, M.: Enhancenet: Single image super-resolution through automated texture synthesis. arXiv preprint arXiv:1612.07919 (2016)

  35. Semmo, A., Limberger, D., Kyprianidis, J.E., Döllner, J.: Image stylization by oil paint filtering using color palettes. In: Proceedings of the Workshop on Computational Aesthetics, CAE ’15, pp. 149–158. Eurographics Association, Goslar Germany, Germany (2015)

  36. Shih, Y., Paris, S., Barnes, C., Freeman, W.T., Durand, F.: Style transfer for headshot portraits. ACM Trans. Graph. 33(4), 148:1–148:14 (2014)

    Article  Google Scholar 

  37. Shih, Y., Paris, S., Durand, F., Freeman, W.T.: Data-driven hallucination of different times of day from a single outdoor photo. ACM Trans. Graph. 32(6), 200:1–200:11 (2013)

    Article  Google Scholar 

  38. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Computer Vision, pp. 839–846. IEEE (1998)

  39. Wang, B., Wang, W., Yang, H., Sun, J.: Efficient example-based painting and synthesis of 2d directional texture. IEEE Trans. Vis. Comput. Graph. 10(3), 266–277 (2004)

    Article  Google Scholar 

  40. Wei, L.Y., Lefebvre, S., Kwatra, V., Turk, G.: State of the Art in Example-based Texture Synthesis. Eurographics 2009. State of the Art Report, EG-STAR, pp. 93–117. Eurographics Association, Munich, Germany (2009)

  41. Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 479–488. ACM Press/Addison-Wesley Publishing Co. (2000)

  42. Wexler, Y., Shechtman, E., Irani, M.: Space–time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007). https://doi.org/10.1109/TPAMI.2007.60

    Article  Google Scholar 

  43. Zhou, Y., Shi, H., Lischinski, D., Gong, M., Kopf, J., Huang, H.: Analysis and controlled synthesis of inhomogeneous textures. Comput. Graph. Forum 36(2), 199–212 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers for many insightful comments. Thanks also to Eric Paquette and members of the Graphics, Imaging and Games Lab for productive comments and discussions. Funding for this work was provided by NSERC and by Carleton University. We used many images from Flickr under a Creative Commons license. Thanks to the numerous photographers who provided material: Bill Showalter (barn), Jim Sorbie (cabin), Alana Sise (rocky hill), Sérgio Sakakibara (cameraman), Harry Rose (grass), delta ! (girl), bananaana04 (parrot), Maria Morri (earring), Donald Lammers (ruin), Tim Adams (walkway), Ryan Basilio (berries), Gábor Lengyel (portrait), Stephanie Kroos (lion), and Martin Pettitt (car).

Author information

Authors and Affiliations

  1. Carleton University, Ottawa, Ontario, Canada

    Lars Doyle & David Mould

Authors
  1. Lars Doyle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Mould
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lars Doyle.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doyle, L., Mould, D. Augmenting photographs with textures using the Laplacian pyramid. Vis Comput 35, 1489–1500 (2019). https://doi.org/10.1007/s00371-018-1513-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1513-y

Keywords

Search

Navigation