research-article

EasyFont: A Style Learning-Based System to Easily Build Your Large-Scale Handwriting Fonts

Authors:

Jianguo XiaoAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 38, Issue 1

Article No.: 6, Pages 1 - 18

https://doi.org/10.1145/3213767

Published: 14 December 2018 Publication History

Abstract

Generating personal handwriting fonts with large amounts of characters is a boring and time-consuming task. For example, the official standard GB18030-2000 for commercial font products consists of 27,533 Chinese characters. Consistently and correctly writing out such huge amounts of characters is usually an impossible mission for ordinary people. To solve this problem, we propose a system, EasyFont, to automatically synthesize personal handwriting for all (e.g., Chinese) characters in the font library by learning style from a small number (as few as 1%) of carefully-selected samples written by an ordinary person. Major technical contributions of our system are twofold. First, we design an effective stroke extraction algorithm that constructs best-suited reference data from a trained font skeleton manifold and then establishes correspondence between target and reference characters via a non-rigid point set registration approach. Second, we develop a set of novel techniques to learn and recover users’ overall handwriting styles and detailed handwriting behaviors. Experiments including Turing tests with 97 participants demonstrate that the proposed system generates high-quality synthesis results, which are indistinguishable from original handwritings. Using our system, for the first time, the practical handwriting font library in a user’s personal style with arbitrarily large numbers of Chinese characters can be generated automatically. It can also be observed from our experiments that recently-popularized deep learning based end-to-end methods are not able to properly handle this task, which implies the necessity of expert knowledge and handcrafted rules for many applications.

Supplementary Material

MP4 File (a6-lian.mp4)

Download
224.41 MB

References

[1]

M. Arjovsky, S. Chintala, and L. Bottou. 2017. Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017).

[2]

S. Baluja. 2016. Learning typographic style. CoRR abs/1603.04000 (2016). http://arxiv.org/abs/1603.04000.

[3]

W. Baxter and N. Govindaraju. 2010. Simple data-driven modeling of brushes. In Proc. ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 135--142.

Digital Library

[4]

E. Bernhardsson. 2016. Analyzing 50k fonts using deep neural networks. Retrieved from https://erikbern.com/2016/01/21/analyzing-50k-fonts-using-deep-neural-networks/.

[5]

N. D. F. Campbell and J. Kautz. 2014. Learning a manifold of fonts. ACM Transactions on Graphics 33, 4 (2014), 91.

Digital Library

[6]

X. Chen, Z. Lian, Y. Tang, and J. Xiao. 2017. An automatic stroke extraction method using manifold learning. In Proc. Eurographics 2017 Short Paper.

[7]

Já. Dolinsky and H. Takagi. 2009. Analysis and modeling of naturalness in handwritten characters. IEEE Transactions on Neural Networks 20, 10 (2009), 1540--1553.

Digital Library

[8]

J. Dong, M. Xu, and Y. Pan. 2008. Statistic model-based simulation on calligraphy creation. Chinese Journal of Computers 31, 7 (2008), 1276--1282 (In Chinese).

[9]

J. L. Elman. 1990. Finding structure in time. Cognitive Science 14, 2 (1990), 179--211.

[10]

J. Fan. 1990a. Intelligent Chinese character design and an experimental system ICCDS. JCIP 4, 3 (1990), 1--11 (In Chinese).

[11]

J. Fan. 1990b. A method of computerizing the calligraphical rules basing on CC structure code. JCIP 4, 4 (1990), 43--52 (In Chinese).

[12]

FontCreator. 2017. High logic. Retrieved from http://www.high-logic.com/.

[13]

FontLab. 2017. Fontlab. Retrieved from http://www.fontlab.com/.

[14]

Founder. 2017. Founder group. Retrieved from http://www.foundertype.com/.

[15]

L. A. Gatys, A. S. Ecker, M. Bethge, S. Hertzmann, and E. Shechtman. 2017. Controlling perceptual factors in neural style transfer. In Proc. CVPR 2017.

[16]

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative adversarial networks. In Proc. NIPS 2014.

Digital Library

[17]

T. S. F. Haines, M. Aodha, and G. J. Brostow. 2016. My text in your handwriting. ACM Transactions on Graphics (TOG) 35, 3 (2016), 26.

Digital Library

[18]

K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In Proc. CVPR 2016. 770--778.

[19]

G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504--507.

[20]

S. Hochreiter and J. Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780.

Digital Library

[21]

P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros. 2017. Image-to-image translation with conditional adversarial nets. In Proc. CVPR 2017.

[22]

B. K. Jang and R. T. Chin. 1990. Analysis of thinning algorithms using mathematical morphology. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 6 (1990), 541--551.

Digital Library

[23]

L. Jin and X. Gao. 2004. Study of several handwritten Chinese character directional feature extraction approaches. Application Research of Computers 21, 11 (2004), 38--40.

[24]

H. Khosravi and E. Kabir. 2010. Farsi font recognition based on Sobel--Roberts features. Pattern Recognition Letters 31, 1 (2010), 75--82.

Digital Library

[25]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25. 1097--1105.

Digital Library

[26]

P. K. Lai, D. Y. Yeung, and M. C. Pong. 1996. A heuristic search approach to Chinese glyph generation using hierarchical character composition. Computer Processing of Oriental Languages 10, 3 (1996), 307--323.

[27]

B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. 2015. Human-level concept learning through probabilistic program induction. Science 350, 6266 (2015), 1332--1338.

[28]

N. Lawrence. 2005. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research 6, Nov (2005), 1783--1816.

Digital Library

[29]

W. Li, Y. Song, and C. Zhou. 2014. Computationally evaluating and synthesizing Chinese calligraphy. Neurocomputing 135, 5 (2014), 299--305.

Digital Library

[30]

Z. Lian and J. Xiao. 2012. Automatic shape morphing for Chinese characters. In Proc. SIGGRAPH Asia 2012 TB. 2.

Digital Library

[31]

Z. Lian, B. Zhao, and J. Xiao. 2016. Automatic generation of large-scale handwriting fonts via style learning. In Proc. SIGGRAPH Asia 2016 TB. 12.

Digital Library

[32]

J. Lin, C. Wang, C. Ting, and R. Chang. 2014. Font generation of personal handwritten Chinese characters. In Proc. IGIP 2014.

[33]

Z. Lin and L. Wan. 2007. Style-preserving English handwriting synthesis. Pattern Recognition 40, 7 (2007), 2097--2109.

Digital Library

[34]

J. Long, E. Shelhamer, and T. Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proc. CVPR 2015. 3431--3440.

[35]

J. Lu, C. Barnes, S. DiVerdi, and A. Finkelstein. 2013. RealBrush: Painting with examples of physical media. In Proc. ACM SIGGRAPH 2013.

Digital Library

[36]

J. Lu, C. Barnes, C. Wan, P. Asente, R. Mech, and A. Finkelstein. 2014. DecoBrush: Drawing structured decorative patterns by example. In Proc. ACM SIGGRAPH 2014.

Digital Library

[37]

J. Lu, F. Yu, A. Finkelstein, and S. DiVerdi. 2012. HelpingHand: Example-based stroke stylization. In Proc. ACM SIGGRAPH 2012.

Digital Library

[38]

A. Myronenko and X. Song. 2010. Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 12 (2010), 2262--2275.

Digital Library

[39]

W. Pan, Z. Lian, R. Sun, Y. Tang, and J. Xiao. 2014. FlexiFont: A flexible system to generate personal font libraries. In Proc. DocEng 2014. 17--20.

Digital Library

[40]

H. Q. Phan, H. Fu, and A. B. Chan. 2015. FlexyFont: Learning transferring rules for flexible typeface synthesis. Computer Graphics Forum 34, 7 (2015), 245--256.

Digital Library

[41]

D. E. Rumelhart, G. E. Hinton, and R. J. Williams. 1986. Learning representations by back-propagating errors. Nature 323, 6088 (1986), 533--536.

[42]

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252.

Digital Library

[43]

D. Silver, A. Huang, and C. J. Maddison. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484--489.

[44]

K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014).

[45]

A. A. Soltani, H. Huang, J. Wu, T. Kulkarni, and J. Tenenbaum. 2017. Synthesizing 3D shapes via modeling multi-view depth maps and silhouettes with deep generative networks. In Proc. CVPR 2017.

[46]

S. Strassmann. 1986. Hairy brushes. In Proc. ACM SIGGRAPH 1986, Vol. 20. 225--232.

Digital Library

[47]

Z. Sun, L. Jin, Z. Xie, Z. Feng, and S. Zhang. 2016. Convolutional multi-directional recurrent network for offline handwritten text recognition. In 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). 240--245.

[48]

R. Suveeranont and T. Igarashi. 2010. Example-based automatic font generation. In Proc. Smart Graphics. 127--138.

Digital Library

[49]

Y. Tian. 2016. ReWrite. Retrieved from https://github.com/kaonashi-tyc/Rewrite/.

[50]

Y. Tian. 2017. ReWrite. Retrieved from https://github.com/kaonashi-tyc/zi2zi/.

[51]

Y. Wang, H. Wang, C. Pan, and L. Fang. 2008. Style preserving Chinese character synthesis based on hierarchical representation of character. In Proc. ICASSP 2008. 1097--1100.

[52]

Z. Wang and Y. Pang. 1991. A computer calligraphy system CCCS. Journal of Computer Aided Design and Computer Graphics 3, 1 (1991), 35--40 (In Chinese).

[53]

S. T. Wong, H. Leung, and H. H. S. Ip. 2008. Model-based analysis of Chinese calligraphy images. Computer Vision and Image Understanding 109, 1 (2008), 69--85.

Digital Library

[54]

W. Xia and L. Jin. 2009. A Kai style calligraphic beautification method for handwriting chinese character. In Proc. ICDAR 2009. 798--802.

Digital Library

[55]

S. Xu, H. Jiang, T. Jin, F. Lau, and Y. Pan. 2008. Automatic facsimile of Chinese calligraphic writings. In Computer Graphics Forum, Vol. 27. 1879--1886.

[56]

S. Xu, H. Jiang, F. C. M. Lau, and Y. Pan. 2007. An intelligent system for Chinese calligraphy. In Proc. The National Conference on Artificial Intelligence, Vol. 22. 1578.

Digital Library

[57]

S. Xu, T. Jin, H. Jiang, and F. C. M. Lau. 2009. Automatic generation of personal chinese handwriting by capturing the characteristics of personal handwriting. In Proc. IAAI 2009.

[58]

S. Xu, F. Lau, F. Tang, and Y. Pan. 2003. Advanced design for a realistic virtual brush. In Computer Graphics Forum, Vol. 22. 533--542.

[59]

S. Xu, F. C. M. Lau, W. K. Cheung, and Y. Pan. 2005. Automatic generation of artistic Chinese calligraphy. IEEE Intelligent Systems 20, 3 (2005), 32--39.

Digital Library

[60]

T. Yi, Z. Lian, Y. Tang, and J. Xiao. 2014. A data-driven personalized digital ink for Chinese characters. In Proc. MultiMedia Modeling 2014. 254--265.

Digital Library

[61]

K. Yu, J. Wu, and Y. Zhuang. 2009. Style-consistency calligraphy synthesis system in digital library. In Proc. the 9th ACM/IEEE-CS Joint Conference on Digital Libraries. 145--152.

Digital Library

[62]

Z. Zhang, C. Zhang, W. Shen, C. Yao, W. Liu, and X. Bai. 2016. Multi-oriented text detection with fully convolutional networks. In Proc. CVPR 2016. 4159--4167.

[63]

B. Zhou, W. Wang, and Z. Chen. 2011. Easy generation of personal chinese handwritten fonts. In Proc. ICME 2011. 1--6.

Digital Library

[64]

C. L. Zitnick. 2013. Handwriting beautification using token means. In Proc. ACM SIGGRAPH 2013.

Digital Library

Cited By

Wang BZhang KChen ZShen LShen XLiu YBian JShen H(2025)An intelligent font generation system based on stroke inference, mitigating production labor and enhancing design experienceExpert Systems with Applications10.1016/j.eswa.2024.125657263(125657)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125657
Liu YLian ZWooldridge MDy JNatarajan S(2024)DeepCalliFontProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i4.28168(3774-3782)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i4.28168
Li HLian Z(2024)HFH-Font: Few-shot Chinese Font Synthesis with Higher Quality, Faster Speed, and Higher ResolutionACM Transactions on Graphics10.1145/368799443:6(1-16)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687994
Show More Cited By

Index Terms

EasyFont: A Style Learning-Based System to Easily Build Your Large-Scale Handwriting Fonts
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Automatic generation of large-scale handwriting fonts via style learning
SA '16: SIGGRAPH ASIA 2016 Technical Briefs

Generating personal handwriting fonts with large amounts of characters is a boring and time-consuming task. Take Chinese fonts as an example, the official standard GB18030-2000 for commercial font products contains 27533 simplified Chinese characters. ...
Two template matching approaches to Arabic, Amharic and Latin isolated characters recognition

With the establishment of commercial OCR systems for Latin text, recent research efforts have been directed at the design of recognition systems for non-Latin scripts, such as Japanese, Cyrillic, Chinese, Hindi, Tibetan, and in particular Arabic. The ...
Touching character segmentation of Devanagari script
ICCCNT '16: Proceedings of the 7th International Conference on Computing Communication and Networking Technologies

Segmentation of characters is one of the major step in OCR system. Devanagari script is a two dimensional form of symbol. It is very inconvenient to treat each form of character as a separate symbol because such combinations are very large in number. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 38, Issue 1

February 2019

176 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3300145

Editor:
Marc Alexa
TU Berlin, Germany

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 December 2018

Accepted: 01 September 2018

Revised: 01 September 2018

Received: 01 June 2017

Published in TOG Volume 38, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Language Committee of China
Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology)
National Natural Science Foundation of China
National Key Research and Development Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
1,272
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)15

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang BZhang KChen ZShen LShen XLiu YBian JShen H(2025)An intelligent font generation system based on stroke inference, mitigating production labor and enhancing design experienceExpert Systems with Applications10.1016/j.eswa.2024.125657263(125657)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125657
Liu YLian ZWooldridge MDy JNatarajan S(2024)DeepCalliFontProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i4.28168(3774-3782)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i4.28168
Li HLian Z(2024)HFH-Font: Few-shot Chinese Font Synthesis with Higher Quality, Faster Speed, and Higher ResolutionACM Transactions on Graphics10.1145/368799443:6(1-16)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687994
Tatsukawa YShen IQi AKoyama YIgarashi TShamir A(2024)FontCLIP: A Semantic Typography Visual‐Language Model for Multilingual Font ApplicationsComputer Graphics Forum10.1111/cgf.1504343:2Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15043
Yao YWang CWang HWang KRen YMeng W(2024)Embedding Secret Message in Chinese Characters via Glyph Perturbation and Style TransferIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.337790319(4406-4419)Online publication date: 19-Mar-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3377903
He XZhu MWang NGao X(2024)Few-Shot Font Generation by Learning Style Difference and SimilarityIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.338262134:9(8013-8025)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3382621
Wei WYan L(2024)Study on the Method of Progressive Restoration of Defective Chinese Characters2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650289(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650289
Thamizharasan VLiu DAgarwal SFisher MGharbi MWang OJacobson AKalogerakis E(2024)VecFusion: Vector Font Generation with Diffusion2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00759(7943-7952)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00759
Chen DTerzidis KFabrocini F(2024)Deep permutation type design for Chinese charactersThe Design Journal10.1080/14606925.2024.237745527:5(867-887)Online publication date: 16-Jul-2024
https://doi.org/10.1080/14606925.2024.2377455
Li STang SLian Z(2024)CHWmaster: mastering Chinese handwriting via sliding-window recurrent neural networksInternational Journal on Document Analysis and Recognition10.1007/s10032-024-00468-927:3(275-288)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10032-024-00468-9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents