Skip to main content
SpringerLink
Log in

CalliGAN: Unpaired Mutli-chirography Chinese Calligraphy Image Translation

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11362))

Included in the following conference series:

Abstract

The style translation of Chinese calligraphy image is a challenging problem: changing the style and layout of strokes while retaining the content attributes like overall structure and combination of radicals. The existing calligraphy character generation methods, such as brush modeling, skeleton rendering, strokes extracting and assembling, etc., are ineffective and difficult to apply to multi-chirography styles and numerous data. The recent neural style transfer methods for general image-to-image style translation tasks can not apply to our problem due to the different meanings of the “style”. The GAN-based methods demonstrating good results require image-pairs for training, which is hard to collect in our task. Therefore, in this paper we propose a novel GAN-based model, called CalliGAN, for the mutli-chirography Chinese calligraphy image translation. In CalliGAN, We present a joint optimization method which only requires unpaired multiple chirography sets for training. In our experiment, we build a chirography style dataset called Chiro-4 and then compare our method with various general translation methods. The experiment results demonstrate the validity of our method on calligraphy style translation task.

Supported by the National Nature Science Foundation of China (No. 61379073), the Fundamental Research Funds for the Central Universities (2018FZA5012), and China Academic Digital Associative Library (CADAL).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.cadal.zju.edu.cn.

References

  1. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)

  2. Choi, Y., Choi, M., Kim, M., Ha, J., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. CoRR abs/1711.09020 (2017). http://arxiv.org/abs/1711.09020

  3. Gao, P., Wu, J., Xia, Y., Lin, Y.: CADAL digital calligraphy system. In: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL 2012, pp. 357–358. ACM, New York (2012)

    Google Scholar 

  4. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. Comput. Sci. (2015)

    Google Scholar 

  5. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  6. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  7. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779 (2017)

    Google Scholar 

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  9. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. CoRR, abs/1703.06868 (2017)

    Google Scholar 

  10. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976. IEEE (2017)

    Google Scholar 

  11. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  12. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Comput. Sci. (2014)

    Google Scholar 

  13. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. Statistics 1050, 10 (2014)

    Google Scholar 

  14. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  15. Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2479–2486 (2016)

    Google Scholar 

  16. Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: Advances in Neural Information Processing Systems, pp. 700–708 (2017)

    Google Scholar 

  17. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  18. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings ICML, vol. 30, p. 3 (2013)

    Google Scholar 

  19. Mirza, M., Osindero, S.: Conditional generative adversarial nets. Comput. Sci., 2672–2680 (2014)

    Google Scholar 

  20. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. Comput. Sci. (2015)

    Google Scholar 

  21. Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). vol. 3, p. 6 (2017)

    Google Scholar 

  22. Strassmann, S.: Hairy brushes. ACM SIGGRAPH Comput. Graph. 20, 225–232 (1986)

    Article  Google Scholar 

  23. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

    Google Scholar 

  24. Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. CoRR abs/1611.02200 (2016). http://arxiv.org/abs/1611.02200

  25. kaonashi tyc: zi2zi: Master chinese calligraphy with conditional adversarial networks. Website (2017). http://github.com/kaonashi-tyc/zi2zi

  26. Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Instance normalization: the missing ingredient for fast stylization. CoRR abs/1607.08022 (2016). http://arxiv.org/abs/1607.08022

  27. Wong, H.T., Ip, H.H.: Virtual brush: a model-based synthesis of Chinese calligraphy. Comput. Graph. 24(1), 99–113 (2000)

    Article  Google Scholar 

  28. Xia, Y., Wu, J., Gao, P., Lin, Y., Mao, T.: Ontology-based model for Chinese calligraphy synthesis. Comput. Graph. Forum 32, 11–20 (2013)

    Article  Google Scholar 

  29. Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318 (2018)

  30. Zhang, Z., Wu, J., Yu, K.: Chinese calligraphy specific style rendering system. In: Proceedings of the 10th Annual Joint Conference on Digital Libraries, pp. 99–108. ACM (2010)

    Google Scholar 

  31. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251. IEEE (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Zhejiang University, College of Computer Science, Hangzhou, China

    Yiming Gao & Jiangqin Wu

Authors
  1. Yiming Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiangqin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiangqin Wu .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C. V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gao, Y., Wu, J. (2019). CalliGAN: Unpaired Mutli-chirography Chinese Calligraphy Image Translation. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20890-5_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20889-9

  • Online ISBN: 978-3-030-20890-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation