Abstract
Existing real-time automatic video abstraction systems rely on local contrast only for identifying perceptually important information and abstract imagery by reducing contrast in low-contrast regions while artificially increasing contrast in higher contrast regions. These methods, however, may fail to accentuate an object against its background for the images with objects of low contrast over background of high contrast. To solve this problem, we propose a progressive abstraction method based on a region-of-interest function derived from an elaborate perception model. Visual contents in perceptually salient regions are emphasized, whereas the background is abstracted appropriately. In addition, the edge-preserving smoothing and line drawing algorithms in this paper are guided by a vector field which describes the flow of salient features of the input image. The whole pipeline can be executed automatically in real time on the GPU, without requiring any user intervention. Several experimental examples are shown to demonstrate the effectiveness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bolz, J., Farmer, I., Grinspun, E., Schröder, P.: Sparse matrix solvers on the GPU: conjugate gradient and multigrid. ACM Trans. Graphics (SIGGRAPH ’03) 22, 3 (2003)
Bousseau, A., Neyret, F., Thollot, J., Salesin, D.: Video watercolorization using bidirectional texture advection. ACM Trans. Graph. (SIGGRAPH ’07) 26, 3 (2007)
Carbral, B., Leedom, L.: Imaging vector fields using line integral convolution. In: Proc. ACM SIGGRAPH ’93 (1993), pp. 263–270
Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. (SIGGRAPH ’07) 26, 3 (2007)
Colbert, M., Reinhard, E., Hughes, C.E.: Painting in high dynamic range. J. Vis. Commun. Image Represent. 18, 5 (2007)
Collomosse, J.P., Hall, P.M.: Cubist style rendering from photographs. IEEE Trans. Vis. Comput. Graph. 9, 4 (2003)
Collomosse, J.P., Rowntree, D., Hall, P.M.: Stroke surfaces: temporally coherent artistic animations from video. IEEE Trans. Vis. Comput. Graph. 11, 5 (2005)
DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. ACM Trans. Graph. (SIGGRAPH ’02) 21, 3 (2002)
Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. (SIGGRAPH ’08) 27, 3 (2008)
Greenspan, H., Belongie, S., Goodman, R., Perona, P., Rakshit, S., Anderson, C.H.: Overcomplete steerable pyramid filters and rotation invariance. In: Proc. IEEE Computer Vision and Pattern Recognition (1994), pp. 222–228
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11 (1998)
Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: Proc. ACM Sym. Non-Photorealistic Animation and Rendering (NPAR ’07). ACM (2007), pp. 43–50
Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. IEEE Trans. Vis. Comput. Graph. 15(1), 62–76 (2009)
Kraus, M., Strengert, M.: Pyramid filters based on bilinear interpolation. In: Proc. International Conf. of Computer Graphics Theory and Applications (GRAPP ’07) (2007), pp. 21–28
Kyprianidis, J.E., Döllner, J.: Image abstraction by structure adaptive filtering. In: Proc. EG UK Theory and Practice of Computer Graphics (2008), pp. 51–58
Lee, C.H., Varshney, A., Jacobs, D.W.: Mesh saliency. ACM Trans. Graph. (SIGGRAPH ’05) 24, 3 (2005)
Lee, S., Kim, G.J., Choi, S.: Real-time tracking of visually attended objects in interactive virtual environments. In: Proc. ACM Sym. Virtual Reality Software and Technology (VRST ’07) (2007), pp. 29–38
McCloud, S.: Understanding Comics. Harper Collins Publishers, New York (1993)
Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: Proc. ACM Sym. Non-Photorealistic Animation and Rendering (NPAR ’07) (2007), pp. 103–110
Pham, T.Q., van Vliet, L.J.: Separable bilateral filtering for fast video preprocessing. In: Proc. IEEE International Conf. Multimedia and Expo (ICME ’05) (2005), pp. 454–457
Rempel, A.G., Trentacoste, M., Seetzen, H., Young, H.D., Heidrich, W., Whitehead, L., Ward, G.: Ldr2hdr: on-the-fly reverse tone mapping of legacy video and photographs. ACM Trans. Graph. (SIGGRAPH ’07) 26, 3 (2007)
Santella, A., DeCarlo, D.: Visual interest and NPR: an evaluation and manifesto. In: Proc. ACM Sym. Non-Photorealistic Animation and Rendering (NPAR ’04) (2004), pp. 71–78
Scheuermann, T., Hensley, J.: Efficient histogram generation using scattering on GPUs. In: Proc. ACM Sym. Interactive 3D Graphics and Games (I3D ’07) (2007), pp. 33–37
Setlur, V., Lechner, T., Nienhaus, M., Gooch, B.: Retargeting images and video for preserving information saliency. IEEE Comput. Graph. Appl. 27, 5 (2007)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. IEEE International Conf. Computer Vision (ICCV ’98) (1998), pp. 839–846
Wang, J., Xu, Y., Shum, H.Y., Cohen, M.F.: Video tooning. ACM Trans. Graph. (SIGGRAPH ’04) 23, 3 (2004)
Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. ACM Trans. Graph. (SIGGRAPH ’06) 25, 3 (2006)
Wyszecki, G., Stiles, W.S.: Color science: concepts and methods, quantitative data and formulae. Wiley, New York (1982)
Zhao, H., Jin, X., Shen, J., Mao, X., Feng, J.: Real-time feature-aware video abstraction. Vis. Comput. (CGI ’08) 24, 7 (2008)
Ziegler, G., Tevs, A., Theobalt, C., Seidel, H.-P.: GPU point list generation through histogram pyramids. In: Proc. 11th Fall Workshop on Vision, Modeling, and Visualization (VMV ’06) (2006), pp. 133–141
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, H., Mao, X., Jin, X. et al. Real-time saliency-aware video abstraction. Vis Comput 25, 973–984 (2009). https://doi.org/10.1007/s00371-008-0308-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-008-0308-y