Shuai Yang
ABSTRACT
In this work, we present a novel algorithm to stylize the text without supervision, which provides a flexible and convenient way to invoke fantastic text expressions. Rather than employing the fixed pair of target text and source style images, our unsupervised framework establishes an implicit mapping for them by using an abstract imagery of the style image as bridges. Based on the mapping, we progressively narrow the visual discrepancy between text and style images by the proposed legibility-preserving structure transfer and texture transfer algorithms, which effectively balance the text legibility and style consistency. Furthermore, we explore a seamless composition of the stylized text and a background image, in which the optimal text layout is determined by a context-aware layout design algorithm utilizing cues for both seamlessness and aesthetics. Given the layout, the text can be seamlessly embedded into the background by texture synthesis under a context-aware boundary constraint. Experimental results demonstrate the effectiveness of the proposed method in automatic artistic typography creation and visual-textual presentation synthesis.
- Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 34, 11 (2012), 2274--2282. Google Scholar
Digital Library
- C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. 2009. PatchMatch: a Randomized Correspondence Algorithm for Structural Image Editing. ACM Transactions on Graphics , Vol. 28, 3 (August 2009), 341--352. Google ScholarDigital Library
- Connelly Barnes, Fang Lue Zhang, Liming Lou, Xian Wu, and Shi Min Hu. 2015. PatchTable: efficient patch queries for large datasets and applications. ACM Transactions on Graphics , Vol. 34, 4 (2015), 97. Google ScholarDigital Library
- Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden. 2013. Stylizing animation by example. ACM Transactions on Graphics , Vol. 32, 4 (2013), 119. Google ScholarDigital Library
- Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. 2000. Image inpainting. ACM Transactions on Graphics (2000), 417--424.Google Scholar
- Alex J. Champandard. 2016. Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks. (2016). arXiv:1603.01768.Google Scholar
- Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, and Gang Hua. 2017a. Coherent Online Video Style Transfer. In Proc. Int'l Conf. Computer Vision .Google ScholarCross Ref
- Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, and Gang Hua. 2017b. StyleBank: An Explicit Representation for Neural Image Style Transfer. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google ScholarCross Ref
- A. Criminisi, P. Pérez, and K. Toyama. 2004. Region Filling and Object Removal by Exemplar-Based Image Inpainting. IEEE Transactions on Image Processing , Vol. 13 (September 2004), 1200 -- 1212. Google ScholarDigital Library
- Alexei A. Efros and William T. Freeman. 2001. Image quilting for texture synthesis and transfer. In Proc. ACM Conf. Computer Graphics and Interactive Techniques. 341--346. Google ScholarDigital Library
- Alexei A Efros and Thomas K Leung. 1999. Texture synthesis by non-parametric sampling. In Proc. IEEE Int'l Conf. Computer Vision . Google ScholarDigital Library
- Michael Elad and Peyman Milanfar. 2017. Style Transfer via Texture Synthesis. IEEE Transactions on Image Processing , Vol. 26, 5 (2017), 2338--2351. Google ScholarDigital Library
- Oriel Frigo, Neus Sabater, Julie Delon, and Pierre Hellier. 2016. Split and Match: example-based Adaptive Patch Sampling for Unsupervised Style Transfer. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google ScholarCross Ref
- Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015. Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems . Google ScholarDigital Library
- Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google ScholarCross Ref
- L. A. Gatys, A. S. Ecker, M. Bethge, A. Hertzmann, and E. Shechtman. 2017. Controlling Perceptual Factors in Neural Style Transfer. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google Scholar
- Tom S. F. Haines, Oisin Mac Aodha, and Gabriel J. Brostow. 2016. My Text in Your Handwriting. ACM Transactions on Graphics , Vol. 35, 3 (May 2016), 26:1--26:18. Google ScholarDigital Library
- Aaron Hertzmann. 2001. Algorithms for rendering in artistic styles . Ph.D. Dissertation. New York University.Google Scholar
- Aaron Hertzmann, Charles E. Jacobs, Nuria Oliver, Brian Curless, and David H. Salesin. 2001. Image analogies. In Proc. Conf. Computer Graphics and Interactive Techniques. 327--340. Google ScholarDigital Library
- Justin Johnson, Alexandre Alahi, and Fei Fei Li. 2016. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In Proc. European Conf. Computer Vision .Google ScholarCross Ref
- Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, and Aaron Bobick. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Transactions on Graphics , Vol. 22, 3 (2003), 277--286. Google ScholarDigital Library
- Chuan Li and Michael Wand. 2016. Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google ScholarCross Ref
- Shaohua Li, Xinxing Xu, Liqiang Nie, and Tat Seng Chua. 2017. Laplacian-Steered Neural Style Transfer. In Proc. ACM Int'l Conf. Multimedia . Google ScholarDigital Library
- Zhouhui Lian, Bo Zhao, and Jianguo Xiao. 2016. Automatic Generation of Large-scale Handwriting Fonts via Style Learning. In SIGGRAPH ASIA 2016 Technical Briefs. ACM, Article 12, bibinfonumpages12:1--12:4 pages. Google ScholarDigital Library
- Lin Liang, Ce Liu, Yingqing Xu, Baining Guo, and Heungyeung Shum. 2001. Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics , Vol. 20, 3 (2001), 127--150. Google ScholarDigital Library
- Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, and Singbing Kang. 2017. Visual attribute transfer through deep image analogy. ACM Transactions on Graphics , Vol. 36, 4 (2017), 120. Google ScholarDigital Library
- Yitzchak David Lockerman, Basile Sauvage, Rémi Allègre, Jean-Michel Dischler, Julie Dorsey, and Holly Rushmeier. 2016. Multi-scale label-map extraction for texture synthesis. ACM Transactions on Graphics , Vol. 35, 4 (2016), 140. Google ScholarDigital Library
- R. Maharik, M. Bessmeltsev, A. Sheffer, A. Shamir, and N. Carr. 2011. Digital Micrography. ACM Transactions on Graphics (2011), 100:1--100:12. Google ScholarDigital Library
- Amir Rosenberger, Daniel Cohen-Or, and Dani Lischinski. 2009. Layered shape synthesis: automatic generation of control maps for non-stationary textures. ACM Transactions on Graphics , Vol. 28, 5 (2009), 107. Google ScholarDigital Library
- Ahmed Selim, Mohamed Elgharib, and Linda Doyle. 2016. Painting style transfer for head portraits using convolutional neural networks. ACM Transactions on Graphics , Vol. 35, 4 (2016), 1--18. Google ScholarDigital Library
- Yichang Shih, Sylvain Paris, Connelly Barnes, William T. Freeman, and Frédo Durand. 2014. Style transfer for headshot portraits. ACM Transactions on Graphics , Vol. 33, 4 (2014), 1--14. Google ScholarDigital Library
- Yichang Shih, Sylvain Paris, Frédo Durand, and William T. Freeman. 2013. Data-driven hallucination of different times of day from a single outdoor photo. ACM Transactions on Graphics , Vol. 32, 6 (2013), 2504--2507. Google ScholarDigital Library
- O. $textLe Meur$, M. Ebdelli, and C. Guillemot. 2013. Hierarchical Super-Resolution-Based Inpainting. IEEE Transactions on Image Processing , Vol. 22 (October 2013), 3779 -- 3790. Google ScholarDigital Library
- Y. Wexler, E. Shechtman, and M. Irani. 2007. Space-Time Completion of Video. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 29, 3 (March 2007), 463--476. Google ScholarDigital Library
- Li Xu, Qiong Yan, Yang Xia, and Jiaya Jia. 2012. Structure extraction from texture via relative total variation. ACM Transactions on Graphics , Vol. 31, 6 (2012), 139. Google ScholarDigital Library
- Xuemiao Xu, Linling Zhang, and Tien-Tsin Wong. 2010. Structure-based ASCII Art. ACM Transactions on Graphics , Vol. 29, 4 (July 2010), 52:1--52:9. Google ScholarDigital Library
- Shuai Yang, Jiaying Liu, Zhouhui Lian, and Zongming Guo. 2017. Awesome Typography: Statistics-Based Text Effects Transfer. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition .Google ScholarCross Ref
- Jianming Zhang and Stan Sclaroff. 2013. Saliency detection: A boolean map approach. In Proc. Int'l Conf. Computer Vision . 153--160. Google ScholarDigital Library
- Shengchuan Zhang, Xinbo Gao, Nannan Wang, and Jie Li. 2015. Face Sketch Synthesis via Sparse Representation-Based Greedy Search. IEEE Transactions on Image Processing , Vol. 24, 8 (2015), 2466--2477.Google ScholarCross Ref
- S Zhang, X Gao, N Wang, and J Li. 2016. Robust Face Sketch Style Synthesis. IEEE Transactions on Image Processing , Vol. 25, 1 (2016), 220--232.Google ScholarCross Ref
- Changqing Zou, Junjie Cao, Warunika Ranaweera, Ibraheem Alhashim, Ping Tan, Alla Sheffer, and Hao Zhang. 2016. Legible Compact Calligrams. ACM Transactions on Graphics , Vol. 35, 4 (July 2016), 122:1--122:12. Google ScholarDigital Library
Index Terms
- Context-Aware Unsupervised Text Stylization
Recommendations
Artistic Text Stylization for Visual-Textual Presentation Synthesis
MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in AsiaIn this research, we study a specific task of visual-textual presentation synthesis, where artistic text is generated and embedded in a background photo. The art form of visual-textual presentation is widely used in graphic design such as posters, ...
Region-based painting style transfer
SA '15: SIGGRAPH Asia 2015 Technical BriefsIn this paper, we present a novel method for creating a painted image from a photograph using an existing painting as a style source. The core idea is to identify the corresponding objects in the two images in order to select patches more appropriately. ...
CLIPTexture: Text-Driven Texture Synthesis
MM '22: Proceedings of the 30th ACM International Conference on MultimediaCan artificial intelligence create textures with artistic value according to human language control? Existing texture synthesis methods require example texture input. However, in many practical situations, users don't have satisfying textures but tell ...
Comments