Abstract
This paper presents a graphics processing unit (GPU) based stylization approach that preserves the fine structure between the original and the stylized images using gradient optimization. Existing abstraction and painterly stylization methods focused on contrast manipulation only, and thus the detailed salient structures of the input images are always destroyed when performing the current stylization techniques because of limitations like unavoidable salience information loss caused by contrast abstraction. We propose an image structure map to naturally model the fine structure existing in the original images. Gradient-based structure tangent generation and tangent-guided image morphology are used to construct the structure map. The image structure map, unlike an edge map, not only systematically models the boundary information within the imagery but also accentuates the underlying inner structure detail for further stylization. We facilitate the final stylization via parallel bilateral grid and structure-aware stylizing optimization on a GPU-CUDA platform in real time. In multiple experiments, the proposed method consistently demonstrates efficient and high quality image stylization performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Lu J W, Sander P V, Finkelstein A. Interactive painterly stylization of images, videos and 3D animations. In: Proc ACM SIGGRAPH Sym Interactive 3D Graphics and Games (I3D’10), 2010. 127–134
Bousseau A, Neyret F, Thollot J, et al. Video watercolorization using bidirectional texture advection. In: Proc SIGGRAPH, 2007
DeCarlo D, Santella A. Stylization and abstraction of photographs. In: Proc SIGGRAPH, 2002
Hertzmann A. Paint by relaxation. In: Proc Computer Graphics International (CGI’01), 2001. 47–54
Zhao M T, Zhu S C. Sisley the abstract painter. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 99–107
Kang H, Lee S Y, Chui C K. Flow-based image abstraction. IEEE Trans Vis Comput Graph, 2009, 15: 62–76
Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. In: Proc SIGGRAPH, 2006
Zhao H L, Mao X Y, Jin X G, et al. Real-time saliency-aware video abstraction. The Visual Computer, 2009, 25: 973–984
Kang H, Lee S Y, Chui C K. Coherent line drawing. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 43–50
Wen F, Luan Q, Liang L, et al. Color sketch generation. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’06), 2006. 47–54
Prendeville B. Realism in 20th Century Painting. New York: Thames & Hudson, 2000
Stremmel K, Grosenick U. Realism. Koln: Taschen, 2004
Hertzmann A. Non-photorealistic rendering and the science of art. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 147–157
Hertzmann A. Painterly rendering with curved brush strokes of multiple sizes. In: Proc SIGGRAPH, 1998. 453–460
Hertzmann A, Jacobs C E, Oliver N, et al. Image Analogies. In: Proc SIGGRAPH, 2001. 327–340
Bousseau A, Kaplan M, Thollot J, et al. Interactive watercolor rendering with temporal coherence and abstraction. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’06), 2006. 141–149
Kyprianidis J E, Kang H. Image and video abstraction by coherence-enhancing filtering. In: Computer Graphics Forum (Eurographics’11), 2011
Kyprianidis J E, Kang H, Döllner J. Image and video abstraction by anisotropic Kuwahara filtering. In: Computer Graphics Forum (Pacific Graphics’09), 2009, 28. 1955–1963
Zhao H L, Jin X G, Shen J B, et al. Real-time feature-aware video abstraction. The Visual Computer, 2008, 24: 727–734
Huang H, Zhang L, Fu T N. Video painting via motion layer manipulation. Computer Graphics Forum, 2010, 29: 2055–2064
Huang H, Fu T N, Li C F. Painterly rendering with content-dependent natural paint strokes. The Visual Computer, 2011, 27: 861–871
Zhang S H, Li X Y, Hu S M, et al. Online video stream abstraction and stylization. IEEE Trans Multimedia, 2011, 13: 1286–1294
Hachisuka T, Jensen H W. Parallel progressive photon mapping on GPUs. In: Proc SIGGRAPH Asia Sketches, 2010
Bertal M, Fort P, Sanchez-Crespo D. Real-time, accurate depth of field using anisotropic diffusion and programmable graphics cards. In: Proc 3DPVT, 2004. 767–773
Kazhdan M, Hoppe H. Streaming multigrid for gradient-domain operations on large images. In: Proc SIGGRAPH, 2008
Orzan A, Bousseau A, Winnemöller H, et al. Diffusion curves: a vector representation for smooth-shaded images. In: Proc SIGGRAPH, 2008
Mavridis P, Papaioannou G. High quality elliptical texture filtering on GPU. In: Proc ACM SIGGRAPH Sym Interactive 3D Graphics and Games (I3D’11), 2011. 23–30
Krähenbühl P, Lang M, Hornung A, et al. A system for retargeting of streaming video. In: Proc SIGGRAPH Asia, 2009
McCann J, Pollard N S. Real-time gradient-domain painting. In: Proc SIGGRAPH, 2008
Paris S, Durand F. A fast approximation of the bilateral filter using a signal processing approach. In: Proc European Conference on Computer Vision (ECCV’06), 2006. 24–52
Chen J, Paris S, Durand F. Real-time edge-aware image processing with the bilateral grid. In: Proc SIGGRAPH, 2007
Cheng M M, Zhang G X, Mitra N J, et al. Global contrast based salient region detection. In: Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), 2011. 409–416
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Li, P., Sun, H., Sheng, B. et al. Image stylization with enhanced structure on GPU. Sci. China Inf. Sci. 55, 1093–1105 (2012). https://doi.org/10.1007/s11432-012-4558-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-012-4558-y