Skip to main content
SpringerLink
Log in

Selective image abstraction

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a novel and convenient method for producing selective stylized simplification of images. The user uses a brush to interactively mark certain areas of the input image which are to be left unaltered. Boundaries of these areas are then automatically optimized to underlying object boundaries in the image. Our method then performs stylized simplification of the unmarked areas, while preserving the marked areas. The method ensures a smooth transition between stylized and unaltered regions to leave a mixed reality image which combines the real and the abstract. Stylized simplification is performed using nonlinear diffusion, which can generate sophisticated results. We modify the classic model of nonlinear diffusion to incorporate bilateral filtering; we apply diffusion speed control of each pixel based on the user’s input. The level of simplification can be controlled intuitively based on the diffusion time; another parameter controls the abstraction style, giving a simple and intuitive user interface. Our contributions include a simple-to-use method to produce a novel NPR style and a modified nonlinear diffusion model suited to this selective stylized simplification task. Experimental results show that the final mixed reality results are harmonious.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alvarez, L., Lions, P.L., Morel, J.M.: Image selective smoothing and edge detection by nonlinear diffusion. II. SIAM J. Numer. Anal. 29(3), 845–866 (1992). doi:10.1137/0729052

    Article  MATH  MathSciNet  Google Scholar 

  2. Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & amp; region segmentation of objects in n–d images. In: Intl. Conf. on Computer Vision, pp. 105–112 (2001)

  3. Canny, F.J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986). doi:10.1145/11274.11275

    Article  Google Scholar 

  4. Catté, F., Lions, P.L., Morel, J.M., Coll, T.: Image selective smoothing and edge detection by nonlinear diffusion. SIAM J. Numer. Anal. 29(1), 182–193 (1992). doi:10.1137/0729012

    Article  MATH  MathSciNet  Google Scholar 

  5. Collomosse, J.P., Rowntree, D., Hall, P.M.: Stroke surfaces: temporally coherent non-photorealistic animations from video. IEEE Trans. Vis. Comput. Graph. 11(5), 540–549 (2005). doi:10.1109/TVCG.2005.85

    Article  Google Scholar 

  6. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  7. DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. In: SIGGRAPH ’02: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 769–776. ACM, New York (2002). doi:10.1145/566570.566650

    Chapter  Google Scholar 

  8. Fischer, J., Bartz, D., Straßer, W.: Stylized augmented reality for improved immersion. In: Proceedings of IEEE Virtual Reality (VR 2005), pp. 195–202 (2005)

  9. Kang, H., Lee, S.: Shape-simplifying image abstraction. Comput. Graph. Forum 27(7), 1773–1780 (2008)

    Article  Google Scholar 

  10. Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: ACM Symposium on Non-Photorealistic Animation and Rendering (NPAR), pp. 43–50 (2007)

  11. Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. IEEE Trans. Vis. Comput. Graph. 15(1), 62–76 (2009). doi:10.1109/TVCG.2008.81

    Article  Google Scholar 

  12. Koenderink, J.J.: The structure of images. Biol. Cybern. 50(5), 363–370 (1984). doi:10.1007/BF00336961

    Article  MATH  MathSciNet  Google Scholar 

  13. Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: International Symposium on Non-Photorealistic Animation and Rendering (NPAR), pp. 103–110 (2007). http://artis.imag.fr/Publications/2007/OBBT07

  14. Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis. 81(1), 24–52 (2009). doi:10.1007/s11263-007-0110-8

    Article  Google Scholar 

  15. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)

    Article  Google Scholar 

  16. Rother, C., Kolmogorov, V., Blake, A.: “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004). doi:10.1145/1015706.1015720

    Article  Google Scholar 

  17. Santella, A., DeCarlo, D.: Visual Interest and NPR: an Evaluation and Manifesto, pp. 71–78. ACM Press, New York (2004). doi:10.1145/987657.987669

    Google Scholar 

  18. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: IEEE International Conference on Computer Vision, p. 839 (1998). doi:10.1109/ICCV.1998.710815

  19. Wang, J., Xu, Y., Shum, H.Y., Cohen, M.F.: Video tooning. ACM Trans. Graph. 23(3), 574–583 (2004). doi:10.1145/1015706.1015763

    Article  Google Scholar 

  20. Wen, F., Luan, Q., Liang, L., Xu, Y.Q., Shum, H.Y.: Color sketch generation. In: NPAR ’06: Proceedings of the 4th International Symposium on Non-photorealistic Animation and Rendering, pp. 47–54. ACM, New York (2006). doi:10.1145/1124728.1124737

    Chapter  Google Scholar 

  21. Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 1221–1226. ACM, New York (2006). doi:10.1145/1179352.1142018

    Chapter  Google Scholar 

  22. Zhang, S.H., Chen, T., Zhang, Y.F., Hu, S.M., Martin, R.R.: Vectorizing cartoon animations. IEEE Trans. Vis. Comput. Graph. 15(4), 618–629 (2009). doi:10.1109/TVCG.2009.9

    Article  Google Scholar 

  23. Zhang, Y.F., Hu, S.M., Martin, R.R.: Shrinkability maps for content-aware video resizing. Comput. Graph. Forum 27(7), 1797–1804 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Lin Cong, Ruofeng Tong & Jinxiang Dong

Authors
  1. Lin Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruofeng Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinxiang Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Cong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cong, L., Tong, R. & Dong, J. Selective image abstraction. Vis Comput 27, 187–198 (2011). https://doi.org/10.1007/s00371-010-0522-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0522-2

Keywords

Search

Navigation