Skip to main content
SpringerLink
Log in

Real-time directional stylization of images and videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper describes a real-time non-photorealistic rendering system that automatically produces stylistic effects from photos and videos. The algorithm includes the capture of salient shapes in the image using a smooth direction field and the abstraction of both colors and shapes along the direction field. In addition to the directional stylization, the proposed algorithm has many good properties. First, the temporal coherence is preserved without any extra processing. Second, the level of abstraction can be controlled within constant time complexity. Last but not least, the parallel GPU implementation allows the real-time stylization of online videos. A variety of experimental results demonstrate the effectiveness of the system in producing high-quality abstract illustrations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Cabral B, Leedom LC (1993) Imaging vector fields using line integral convolution. In: Proc SIGGRAPH ’93. ACM, New York, pp 263–270

    Chapter  Google Scholar 

  2. Chen J, Paris S, Durand F (2007) Real-time edge-aware image processing with the bilateral grid. In: Proc SIGGRAPH ’07. ACM, New York, article 103

    Google Scholar 

  3. Collomosse JP, Rowntree D, Hall PM (2005) Stroke surfaces: temporally coherent artistic animations from video. IEEE Trans Vis Comput Graph 11(5):540–549

    Article  Google Scholar 

  4. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  5. DeCarlo D, Santella A (2002) Stylization and abstraction of photographs. In: Proc SIGGRAPH ‘02. ACM, New York, pp 769–776

    Chapter  Google Scholar 

  6. Hertzmann A (2010) Non-photorealistic rendering and the science of art. In: Proc international symposium on non-photorealistic animation and rendering. ACM, New York, pp 147–157

    Google Scholar 

  7. Kang H, Lee S (2008) Shape-simplifying image abstraction. Comput Graph Forum 27(7):1773–1780

    Article  Google Scholar 

  8. Kang H, Lee S, Chui CK (2007) Coherent line drawing. In: Proc international symposium on non-photorealistic animation and rendering. ACM, New York, pp 43–50

    Chapter  Google Scholar 

  9. Kang H, Lee S, Chui CK (2009) Flow-based image abstraction. IEEE Trans Vis Comput Graph 15(1):62–76

    Article  Google Scholar 

  10. Kuwahara M, Hachimura K, Ehiu S, Kinoshita M (1976) Processing of ri-angiocardiographic images. In: Digital processing of biomedical images. Plenum Press, pp 187–203

  11. Kyprianidis JE, Kang H, Döllner J (2009) Image and video abstraction by anisotropic kuwahara filtering. Comput Graph Forum 28(7):1955–1963

    Article  Google Scholar 

  12. Kyprianidis JE, Kang H (2011) Image and video abstraction by coherence-enhancing filtering. Comput Graph Forum 30(2):593–602

    Article  Google Scholar 

  13. Lu S, Jaffer A, Jin X, Zhao H, Mao X (2011) Mathematical marbling. IEEE Comput Graph Appl. doi:10.1109/MCG.2011.51

    Google Scholar 

  14. NVIDIA Corporation (2010) Cuda programming guide for cuda toolkit 3.0. http://developer.nvidia.com/object/gpucomputing.html

  15. Papari G, Petkov N, Campisi P (2007) Artistic edge and corner enhancing smoothing. IEEE Trans Image Process 16(10):2449–2662

    Article  MathSciNet  Google Scholar 

  16. Pham TQ, Vliet LJ (2005) Separable bilateral filtering for fast video preprocessing. In: Proc international conference on multimedia & expo. IEEE, Washington, pp 1–4

  17. Recherche ED, Automatique EE, Lucioles RD, Deriche R, Deriche R (1992) Recursively implementing the gaussian and its derivatives. In: Proc international conference on image processing. IEEE, Wanhington, pp 263–267

  18. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proc international conference on computer vision. IEEE, Washington, pp 839–846

  19. Wang J, Xu Y, Shum HY, Cohen MF (2004) Video tooning. ACM Trans Graph 23(3):574–583

    Article  Google Scholar 

  20. Weickert J (1998) Anisotropic diffusion in image processing. In: ECMI series. Teubner-Verlag, Stuttgart

  21. Winnemöller H, Olsen SC, Gooch B (2006) Real-time video abstraction. ACM Trans Graph 25(3):1221–1226

    Article  Google Scholar 

  22. Wyszecki G, Stiles WS (2000) Color science: concepts and methods, quantitative data and formulae. In: Wiley series in pure and applied optics, 2nd edn. Wiley-Interscience

  23. Zeng K, Zhao M, Xiong C, Zhu SC (2009) From image parsing to painterly rendering. ACM Trans Graph 29(1):1–11

    Article  Google Scholar 

  24. Zhao H, Jin X, Shen H, Mao X, Feng J (2008) Real-time feature-aware video abstraction. Vis Comput 24(7):727–734

    Article  Google Scholar 

  25. Zhao H, Jin X, Shen J, Shen L, Pan R (2009) Fast shape-simplifying image abstraction using graphics hardware. In: Proc edutainment ’09. Springer-Verlag, Berlin, pp 390–398

  26. Zhao H, Mao X, Jin X, Shen J, Wei F, Feng J (2009) Real-time saliency-aware video abstraction. Vis Comput 25(11):973–984

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their helpful and constructive comments and suggestions in improving the paper. Many thanks also to Kyprianidis for letting the test images available. This work was supported by the National Natural Science Foundation of China (Grant No. 61100146) and the Zhejiang Provincial Natural Science Foundation of China (Grant No. Y1110004). Xiaogang Jin was supported by the NSFC-MSRA Joint Funding (Grant No. 60970159) and the National Natural Science Foundation of China (Grants No. 60933007 and No. 60833007).

Author information

Authors and Affiliations

  1. Intelligent Information Systems Institute, Wenzhou University, Wenzhou, 325035, China

    Hanli Zhao

  2. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, 310058, China

    Xiaogang Jin

  3. University of Yamanashi, Kofu, Japan

    Xiaoyang Mao

Authors
  1. Hanli Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyang Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanli Zhao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhao, H., Jin, X. & Mao, X. Real-time directional stylization of images and videos. Multimed Tools Appl 63, 647–661 (2013). https://doi.org/10.1007/s11042-011-0890-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0890-2

Keywords

Search

Navigation