Abstract
As one of the most successful multimedia tools for digital media and creative industry, computer-aided drawing system assists users to convert the input real photos into painterly style images. Nowadays, it is widely developed as cloud brush engine service in many creative software tools and applications of artistic rendering such as Prisma, Photoshop Cloud, and Meitu, because the machine learning server has more powerful than the stand-alone version. In this paper, we propose a web collaborative Stroke-based Learning and Rendering (WebSBLR) system. Different from the existing methods that are mainly focused on the artistic filters, we concentrate on the stroke realistic rendering engine for browser on client using WebGL and HTML5. Moreover, we implement the learning-based stroke drawing path generation module on the server. By this way, we enable to achieve the computer-supported cooperative work (CSCW), especially for multi-screen synchronous interaction. The experiments demonstrated our method are efficient to web-based multi-screen painting simulation.It can successfully learn artists’ styles and render pictures with consistent and smooth brush strokes.
Similar content being viewed by others
References
Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In: Machine learning, proceedings of the twenty-first international conference(ICML 2004). Banff, Alberta, Canada, July 4-8 2004
Baran I, Lehtinen J, Popovic J (2010) Sketching clothoid splines using shortest paths. Comput Graph Forum 29(2):655–664
Bousseau A, Neyret F, Thollot J, Salesin D (2007) Video watercolorization using bidirectional texture advection. ACM Trans Graph 26(3):104
Chu NS, Tai C (2004) Real-time painting with an expressive virtual chinese brush. IEEE Comput Graph Appl 24(5):76–85
Chu NS, Tai C (2005) Moxi: real-time ink dispersion in absorbent paper. ACM Trans Graph 24(3):504–511
Chu N, Baxter B, Wei LY, Govindaraju NK (2010) Detail-preserving paint modeling for 3D brushes Proceedings of the 8th international symposium on NPAR. ACM, New York, pp 27–34
Davies E (2005) Machine vision: theory algorithms, practicalities, 3rd edn. Elsevier
Eric KJ, Kang H (2011) Image and video abstraction by coherence enhancing filtering. Comput Graph Forum, 593–602
Hertzmann A (2003) A survey of stroke-based rendering. IEEE Comput Graph Appl 23:70–81
Hu MQ, Yang Y, Shen F, Xie N, Shen HT (2018) Hashing with angular reconstructive embeddings. IEEE Trans Image Process 27(2):545–555
Huang Z, Han L, Gong GH (2015) A local adaptive catmull-rom to reduce numerical dissipation of semi-lagrangian advection. J Visual Comp Animat 26 (2):141–146
Jolliffe IT (2002) Principal component analysis. Springer Verlag, New York
Kalogerakis E, Nowrouzezahrai D, Breslav S, Hertzmann A (2012) Learning hatching for pen-and-ink illustration of surfaces. ACM Trans Graph 31(1):1,1–1,17
Kang H, Lee S (2008) Shape-simplifying image abstraction. Comput Graph Forum 27(7):1773–1780
Kyprianidis JE, Kang H, Döllner J (2009) Image and video abstraction by anisotropic kuwahara filtering. Comput Graph Forum 28(7):1955–1963
Kyprianidis JE, Semmo A, Kang H, Döllner J (2010) Anisotropic kuwahara filtering with polynomial weighting functions. In: Theory and practice of computer graphics. Sheffield, United Kingdom, 2010. Proceedings, pp 25–30
Lu JW, Yu F, Finkelstein A, DiVerdi S (2012) Helpinghand: example-based stroke stylization. ACM Trans Graph 31(4):46,1–46,10
Lu J, Barnes C, Diverdi S, Finkelstein A (2013) Realbrush: painting with examples of physical media. Acm Trans Graph 32(4):117
Orbay G, Kara LB (2011) Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans Vis Comput Graph 17(5):694–708
Pham TQ, van Vliet LJ (2005) Separable bilateral filtering for fast video preprocessing. In: Proceedings of the 2005 IEEE international conference on multimedia and expo, ICME 2005, July 6-9, 2005. Amsterdam, pp 454–457
Sehnke F, Osendorfer C, Ru̇ckstieß T, Graves A, Peters J, Schmidhuber J (2010) Parameter-exploring policy gradients. Neur Netw 23(4):551–559
Semmo A, Trapp M, Dürschmid T, Döllner J, Pasewaldt S (2016) Interactive multi-scale oil paint filtering on mobile devices. In: SIGGRAPH ’16. Anaheim, CA, USA, July 24-28, 2016, pp 42:1– 42:2
Shen F, Xu Y, Liu L, Yang Y, Huang Z, Shen HT (2018) Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
Sýkora D, Buriánek J, ára J (2005) Colorization of black-and-white cartoons. Image Vis Comput 23(9):767–782
Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256
Xie N, Laga H, Saito S, Nakajima M (2010) IR2s: interactive real photo to sumi-e. In: Proceedings of the 8th international symposium on non-photorealistic animation and rendering 2010. Annecy, France, June 7-10, 2010, pp 63–71
Xie N, Hachiya H, Sugiyama M (2012) Artist agent: a reinforcement learning approach to automatic stroke generation in oriental ink painting. In: Proceedings of the 29th international conference on machine learning, ICML 2012. Edinburgh, Scotland, UK, June 26 - July 1 2012
Xu S, Xu Y, Kang SB, Salesin D, Pan Y, Shum H (2006) Animating Chinese paintings through stroke-based decomposition. ACM Trans Graph 25(2):239–267
Zeng K, Zhao M, Xiong C, Zhu SC (2010) From image parsing to painterly rendering. ACM Trans Graph 29(1):2,1–2,11
Zhao T, Hachiya H, Niu G, Sugiyama M (2011) Analysis and improvement of policy gradient estimation. Adv Neural Inf Process Syst 24:262–270
Zhao Y, Jin X, Xu Y, Zhao H, Ai M, Zhou K (2015) Parallel style-aware image cloning for artworks. IEEE Trans Visual Comput Graph 21(2):229
Zitnick CL (2013) Handwriting beautification using token means. ACM Trans Graph 32(4):53,1–53,8
Acknowledgments
We firstly thank anonymous reviewers for their helpful comments. This work was supported in part by the National Natural Science Foundation of China under Project 61602088, Project 61572108, Project 61632007, and Project 61502339, the National Thousand-Young-Talents Program of China, the Fundamental Research Funds for the Central Universities under Project ZYGX2014Z007, Project ZYGX2015J055 and Project ZYGX2016J212.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Xie, N., Zhao, T., Yang, Y. et al. Web-based SBLR method of multimedia tools for computer-aided drawing. Multimed Tools Appl 78, 799–816 (2019). https://doi.org/10.1007/s11042-018-5949-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-5949-x