Skip to main content
SpringerLink
Log in

SKFont: skeleton-driven Korean font generator with conditional deep adversarial networks

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

In our research, we study the problem of font synthesis using an end-to-end conditional deep adversarial network with a small sample of Korean characters (Hangul). Hangul comprises of 11,172 characters and is composed by writing in multiple placement patterns. Traditionally, font design has required heavy-loaded human labor, easily taking one year to finish one style set. Even with the help of programmable approaches, it still takes a long time and cannot escape the limitations around the freedom to change parameters. Many trials have been attempted in deep neural network areas to generate characters without any human intervention. Our research focuses on an end-to-end deep learning model, the Skeleton-Driven Font generator (SKFont): when given 114 samples, the system automatically generates the rest of the characters in the same given font style. SKFont involves three steps: First, it generates complete target font characters by observing 114 target characters. Then, it extracts the skeletons (structures) of the synthesized characters obtained from the first step. This process drives the system to sustain the main structure of the characters throughout the whole generation processes. Finally, it transfers the style of the target font onto these learned structures. Our study resolves long overdue shortfalls such as blurriness, breaking, and a lack of delivery of delicate shapes and styles by using the ‘skeleton-driven’ conditional deep adversarial network. Qualitative and quantitative comparisons with the state-of-the-art methods demonstrate the superiority of the proposed SKFont method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://software.naver.com/software/fontList.nhn?categoryId=I0000000.

References

  1. Isola, P., Zhu, J., Zhou, T., Efros. A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)

  2. Mirza, M., Osindero, S.: Conditional Generative Adversarial Nets (2014). arXiv preprint arXiv:1411.1784

  3. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of IEEE, pp. 2278–2324 (1998)

  4. Tian, Y.: Rewrite: neural style transfer for Chinese fonts (2016). https://github.com/kaonashityc/Rewrite

  5. Gatys, L.: Image style transfer using convolutional neural networks. In: CVPR (2016)

  6. Atarsaikhan, G., Iwana, B.K., Narusawa, A., Yanai, K., Uchida, S.: Neural font style transfer. In: Proceedings of the 14th International Conference 25 on Document Analysis and Recognition (ICDAR), vol. 5, pp. 51–56 (2017)

  7. Hayashi, H., Abe, K., Uchida, S.: GlyphGAN: style-consistent font generation based on generative adversarial networks. In: Knowledge-Based Systems, vol. 186 (2019)

  8. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv preprint arXiv:1511.06434

  9. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)

  10. Tian, Y.: zi2zi: Master Chinese calligraphy with conditional adversarial networks (2017). https://github.com/kaonashi-tyc/zi2zi

  11. Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: Proceedings of the 34th International Conference on Machine Learning, pp. 2642–2651 (2017)

  12. Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings

  13. Jiang, Y., Lian, Z., Jianguo, Y., Xiao, J.: DCFont: an end-to-end deep Chinese font generation system, SIGGRAPH Asia, p. 22, TB (2017)

  14. Chang, B., Zhang, Q., Pan, S., Meng, L.: Generating handwritten Chinese characters using cyclegan (2018). CoRR. arXiv:1801.08624

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

  16. Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: CVPR (2017)

  17. Sun, H.: Luo, Y., Ziang , L.: Unsupervised Typography Transfer (2018)

  18. Guo, Y., Lian, Z., Tang, Y., Xiao, J.: Creating new Chinese fonts based on manifold learning and adversarial networks. In: Diamanti, O., Vaxman, A. (eds.) Proceedings of the Eurographics—Short Papers. The Eurographics Association (2018)

  19. Jiang, Y., Lian, Z., Tang, Y., Xiao, J.: SCFont: Structureguided Chinese Font Generation via Deep Stacked Networks (2019)

  20. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: Evolution of optical flow estimation with deep networks. In: CVPR, vol. 2, no. 6 (2017)

  21. Panichev, O., Voloshyna, A.: U-net based convolutional neural network for skeleton extraction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June (2019)

  22. Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 236–239 (1984)

  23. Guo, Z., Hall, R.W.: Parallel thinning with two-subiteration algorithms. Commun. ACM 32, 359–373 (1989)

  24. Lee, T.C., Kashyap, R.L., Chu, C.N.: Building skeleton models via 3-D medial surface/axis thinning algorithms. Comput. Vis. Graph. Image Process. 56, 462–478 (1994)

  25. Lian, Z., et al.: EasyFont: a style learning-based system to easily build your large-scale handwriting fonts. ACM Trans. Graph. 38, 61–618 (2019)

    Article  Google Scholar 

  26. Tang, S., et al.: FontRNN: generating large-scale Chinese fonts via recurrent neural network. Comput. Graph. Forum 38, 567–577 (2019)

  27. Lopes, R.G., et al.: A learned representation for scalable vector graphics. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7929–7938 (2019)

  28. Sun, D. et al.: Learning to write stylized Chinese characters by reading a handful of examples. In: IJCAI (2018)

  29. Baluja, S.: Learning typographic style (2016). arXiv:1603.04000

  30. Azadi, S. et al.: Multi-content GAN for few-shot font style transfer. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7564–7573 (2018)

  31. Gao, Y., et al.: Artistic glyph image synthesis via one-stage few-shot learning. ACM Trans. Graph. (TOG) 38, 1–12 (2019)

    Google Scholar 

  32. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, Bi., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS (2014)

  33. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: MICCAI, pp. 234–241 (2015)

  34. Kurach, K., Lucic, M., Zhai, X., Michalski, M.: The GAN landscape: losses, architectures, regularization, and normalization. In: International Conference on Learning Representations (2019)

  35. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, in PMLR, vol. 37, pp. 448–456 (2015)

  36. Maas, A., Hannun, A., Ng, A.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30 (2013)

  37. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network (2015). arXiv preprint arXiv:1505.00853

  38. Eck, P.: Handwritten Korean character recognition with tensorflow and android (2017). https://github.com/IBM/tensorflow-hangul-recognition

Download references

Funding

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2016-0-00166).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Soongsil University, Seoul, Korea

    Debbie Honghee Ko, Ammar Ul Hassan, Jungjae Suk & Jaeyoung Choi

Authors
  1. Debbie Honghee Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ammar Ul Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jungjae Suk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaeyoung Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaeyoung Choi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 About Hangul

There are 11,172 Korean (Hangul) syllables. They can be constructed in six ways (Fig. 9). The syllable blocks are arranged in phonetic order, the initial (Chosung), medial (Joongsung), and final (Jongsung).

There are 19 consonants (14 singles \(+\) 5 doubles) for Chosung, 21 vowels (10 basics \(+\) 11 combined) for Joongsung, and 27 consonants (14 basics \(+\) 11 combined \(+\) 2 double) for Jongsung; Here, “single” indicates one consonant, “double” indicates a doubled consonant, and “combined” indicates two different consonants (Table 3).

Fig. 9
figure 9

Hangul placements and their examples

Full size image
Table 3 19 Consonants and 21 vowels for Hangul
Full size table

1.2 More comparison results with other models

See Figs. 10, 11 and 12.

Fig. 10
figure 10

Style 1: Arita-buriM

Full size image
Fig. 11
figure 11

Style 2: BinggraeTaomB

Full size image
Fig. 12
figure 12

Style 3: DXMyeongjo

Full size image

1.3 Generating stylized font styles

We also fine-tuned the proposed model to synthesize cursive and pixel based stylized font styles. As shown in figure below our model can synthesize these font styles in a decent quality although these kind of font styles were not used in the pre-training phase (Fig. 13).

Fig. 13
figure 13

Cursive and Pixel font style generation

Full size image

1.4 More qualitative and quantitative results of the proposed SKFont

We synthesized various font styles from the proposed SKFont method and evaluated the generated images from visual and performance metrics perspective. We generated various font styles from which 7 are visually displayed in Fig. 14.

Fig. 14
figure 14

Font characters generated in various font styles by SKFont

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ko, D.H., Hassan, A.U., Suk, J. et al. SKFont: skeleton-driven Korean font generator with conditional deep adversarial networks. IJDAR 24, 325–337 (2021). https://doi.org/10.1007/s10032-021-00374-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-021-00374-4

Keywords

Search

Navigation