Skip to main content
SpringerLink
Log in

Real-time coherent stylization for augmented reality

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a new stylized augmented reality (AR) framework which can generate line drawing and abstracted shading styles. In comparison with the state-of-art work, our framework can significantly improve both the visual immersion of a single frame and the temporal coherence of augmented video streams in real time. In our framework, we first render virtual objects over the input camera images and then uniformly process the combined contents with stylization techniques. For generating line drawing stylization, we first propose a specially designed shading method to render the virtual objects, and then use an adapted Flow-based anisotropic Difference-of-Gaussion (FDoG) filter to yield the high-quality line drawing effect. For generating the abstracted stylization, a focus-guided diffusion filter and a soft color quantization operator are sequentially applied to the augmented image, and then the processed result is combined with the detected edges to produce the final abstraction effect. The presented algorithms are all sympathetic to highly parallel processing, allowing a real-time performance on contemporary graphics hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: SIGGRAPH ’93: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 263–270. ACM, New York (1993)

    Chapter  Google Scholar 

  2. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Article  Google Scholar 

  3. Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: SIGGRAPH ’07: ACM SIGGRAPH 2007 papers, p. 103. ACM, New York (2007)

    Google Scholar 

  4. Chen, J., Turk, G., MacIntyre, B.: Watercolor inspired non-photorealistic rendering for augmented reality. In: VRST ’08: Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology, pp. 231–234. ACM, New York (2008)

    Chapter  Google Scholar 

  5. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., Singh, M.: How well do line drawings depict shape?. In: SIGGRAPH ’09: ACM SIGGRAPH 2009 papers, pp. 1–9. ACM, New York (2009)

    Google Scholar 

  6. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S.: Interactive rendering of suggestive contours with temporal coherence. In: NPAR ’04: Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering, pp. 15–145. ACM, New York (2004)

    Chapter  Google Scholar 

  7. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. In: SIGGRAPH ’03: ACM SIGGRAPH 2003 Papers, pp. 848–855. ACM, New York (2003)

    Chapter  Google Scholar 

  8. DeCarlo, D., Rusinkiewicz, S.: Highlight lines for conveying shape. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering, pp. 63–70. ACM, New York (2007)

    Google Scholar 

  9. Fischer, J., Bartz, D.: Real-time cartoon-like stylization of AR video streams on the GPU. Technical Report WSI-2005-18, Wilhelm Schickard Institute for Computer Science, University of Tübingen (2005)

  10. Fischer, J., Bartz, D.: Stylized augmented reality for improved immersion. In: VR ’05: Proceedings of the 2005 IEEE Conference on Virtual Reality, pp. 195–202, 325. IEEE Computer Society, Washington (2005)

    Chapter  Google Scholar 

  11. Fischer, J., Bartz, D., Straßer, W.: Artistic reality: Fast brush stroke stylization for augmented reality. In: VRST ’05: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 155–158. ACM, New York (2005)

    Chapter  Google Scholar 

  12. Fischer, J., Cunningham, D., Bartz, D., Wallraven, C., Bülthoff, H., Strasser, W.: Measuring the discernability of virtual objects in conventional and stylized augmented reality. In: Eurographics Symposium on Virtual Environments (EGVE), pp. 53–61 (2006)

  13. Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23(1), 27–44 (2004)

    Article  Google Scholar 

  14. Haller, M.: Photorealism or/and non-photorealism in augmented reality. In: VRCAI ’04: Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 189–196. ACM, New York (2004)

    Chapter  Google Scholar 

  15. Hertzmann, A., Zorin, D.: Illustrating smooth surfaces. In: SIGGRAPH ’00: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 517–526. ACM/Addison-Wesley, New York (2000)

    Chapter  Google Scholar 

  16. Judd, T., Durand, F., Adelson, E.H.: Apparent ridges for line drawing. ACM Trans. Graph. 26(3), 19 (2007)

    Article  Google Scholar 

  17. Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, pp. 43–50. ACM, New York (2007)

    Google Scholar 

  18. Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. IEEE Trans. Vis. Comput. Graph. 15(1), 62–76 (2009)

    Article  Google Scholar 

  19. Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IWAR ’99: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality, p. 85 (1999)

  20. Kolomenkin, M., Shimshoni, I., Tal, A.: Demarcating curves for shape illustration. ACM Trans. Graph. 27(5), 157 (2008)

    Article  Google Scholar 

  21. Lee, Y., Markosian, L., Lee, S., Hughes, J.F.: Line drawings via abstracted shading. In: SIGGRAPH ’07: ACM SIGGRAPH 2007 papers, p. 18. ACM, New York (2007)

    Google Scholar 

  22. Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. B, Biol. Sci. 207(1167), 187–217 (1980)

    Article  Google Scholar 

  23. Ohtake, Y., Belyaev, A.G., Seidel, H.P.: Ridge-valley lines on meshes via implicit surface fitting. ACM Trans. Graph. 23(3), 609–612 (2004)

    Article  Google Scholar 

  24. Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, pp. 103–110. ACM, New York (2007)

    Google Scholar 

  25. Pham, T.Q., van Vliet, L.J.: Separable bilateral filtering for fast video preprocessing. In: ICME, pp. 454–457 (2005)

  26. Saito, T., Takahashi, T.: Comprehensible rendering of 3-d shapes. In: SIGGRAPH ’90: Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques, pp. 197–206. ACM, New York (1990)

    Chapter  Google Scholar 

  27. Son, M., Kang, H., Lee, Y., Lee, S.: Abstract line drawings from 2d images. In: PG ’07: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 333–342. IEEE Computer Society, Washington (2007)

    Google Scholar 

  28. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV ’98: Proceedings of the Sixth International Conference on Computer Vision, pp. 839–846. IEEE Computer Society, Washington (1998)

    Google Scholar 

  29. Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 1221–1226. ACM, New York (2006)

    Chapter  Google Scholar 

  30. Zhao, H., Jin, X., Shen, J., Mao, X., Feng, J.: Real-time feature-aware video abstraction. Vis. Comput. 24(7–9), 727–734 (2008)

    Article  Google Scholar 

  31. Zhu, L., Wang, C., Zhu, G., Han, B., Wang, H., Huang, P., Wu, E.: Image spatial diffusion on GPUs. In: IEEE Asia Pacific Conference on Circuits and Systems (APCCAS 2008), pp. 610–613 (2008)

Download references

Author information

Authors and Affiliations

  1. State Key Lab. of CS, Inst. of Software, Chinese Academy of Sciences, Beijing, China

    Shandong Wang, Jian Lu, Xuehui Liu & Enhua Wu

  2. Graduate University of Chinese Academy of Sciences, Beijing, China

    Shandong Wang

  3. Thomson C.R., Beijing, China

    Kangying Cai

  4. University of Macau, Macau, China

    Enhua Wu

Authors
  1. Shandong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kangying Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuehui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Enhua Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shandong Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, S., Cai, K., Lu, J. et al. Real-time coherent stylization for augmented reality. Vis Comput 26, 445–455 (2010). https://doi.org/10.1007/s00371-010-0436-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0436-z

Keywords

Search

Navigation