Abstract
Online handwritten Chinese character generation is an interesting task which has gained more and more attention in recent years. Most of the previous methods are based on autoregressive models, where the trajectory points of characters are generated sequentially. However, this often makes it difficult to capture the global structure of the handwriting data. In this paper, we propose a novel generative model, named Diff-Writer, which can not only generate the specified Chinese characters in a non-autoregressive manner but also imitate the calligraphy style given a few style reference samples. Specifically, Diff-Writer is based on conditional Denoising Diffusion Probabilistic Models (DDPM) and consists of three modules: character embedding dictionary, style encoder, and an LSTM denoiser. The character embedding dictionary and the style encoder are adopted to model the content information and the style information respectively. The denoiser iteratively generates characters using the content and style codes. Extensive experiments on a popular dataset (CASIA-OLHWDB) show that our model is capable of generating highly realistic and stylized Chinese characters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: ICLR (2014)
Goodfellow, I., et al.: Generative adversarial nets. In: NeurIPS (2014)
Xu, P., Hospedales, T.M., Yin, Q., Song, Y.Z., Xiang, T., Wang, L.: Deep learning for free-hand sketch: a survey. TPAMI 45(1), 285–312 (2022)
Aksan, E., Pece, F., Hilliges, O.: DeepWriting: making digital ink editable via deep generative modeling. In: SIGCHI (2018)
Ribeiro, L.S.F., Bui, T., Collomosse, J., Ponti, M.: Sketchformer: transformer-based representation for sketched structure. In: CVPR (2020)
Xie, C., Lai, S., Liao, Q., Jin, L.: high performance offline handwritten Chinese text recognition with a new data preprocessing and augmentation pipeline. In: Bai, X., Karatzas, D., Lopresti, D. (eds.) DAS 2020. LNCS, vol. 12116, pp. 45–59. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57058-3_4
Kang, L., Riba, P., Rusinol, M., Fornes, A., Villegas, M.: Content and style aware generation of text-line images for handwriting recognition. In: TPAMI (2022)
Lai, S., Jin, L., Zhu, Y., Li, Z., Lin, L.: SynSig2Vec: forgery-free learning of dynamic signature representations by Sigma Lognormal-based synthesis and 1D CNN. In: TPAMI (2021)
Ha, D: Recurrent net dreams up fake Chinese characters in vector format with tensorflow. http://blog.otoro.net/2015/12/28/recurrent-netdreams-up-fake-chinese-characters-in-vectorformat-with-tensorflow/
Zhang, X.Y., Yin, F., Zhang, Y.M., Liu, C.L., Bengio, Y.: Drawing and recognizing Chinese characters with recurrent neural network. In: TPAMI, Yoshua Bengio (2018)
Tang, S., Xia, Z., Lian, Z., Tang, Y. and Xiao, J.: FontRNN: generating large-scale Chinese fonts via recurrent neural network. In: CGF (2019)
Zhao, B., Tao, J., Yang, M., Tian, Z., Fan, C., Bai, Y.: Deep imitator: handwriting calligraphy imitation via deep attention networks. In: PR (2020)
Tang, S., Lian, Z.: Write like you: synthesizing your cursive online Chinese handwriting via metric-based meta learning. In: CGF (2021)
Dai, G., et al.: Disentangling writer and character styles for handwriting generation. In: CVPR, Zhuoman Liu and Shuangping Huang (2023)
Das, A., Yang, Y., Hospedales, T., Xiang, T., Song, Y.Z.: ChiroDiff: modelling chirographic data with diffusion models. In: ICLR (2023)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)
Yun, X.L., Zhang, Y.M., Yin, F., and Liu, C.L.: Instance GNN: a learning framework for joint symbol segmentation and recognition in online handwritten diagrams. In: TMM (2021)
Graves, A.: Generating sequences with recurrent neural networks. In: ArXiv (2013)
Ha, D., Eck, D.: A neural representation of sketch drawings. In: ICLR (2018)
Aksan, E., Deselaers, T., Tagliasacchi, A., Hilliges, O.: CoSE: compositional stroke embeddings. In: NeurIPS (2020)
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: ICML (2015)
Dhariwal, P.: Alexander quinn nichol: diffusion models beat GANs on image synthesis. In: NeurIPS (2021)
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: ICLR (2021)
Luo, C.: Understanding diffusion models: a unified perspective. In: arXiv (2022)
Liu, C.L., Yin, F., Wang, D.H., Wang, Q.F.: CASIA online and offline Chinese handwriting databases. In: ICDAR, Qiu-Feng Wang (2011)
Acknowledgements
This work is supported by the Major Project for New Generation of AI under Grant No. 2018AAA0100400 and the National Natural Science Foundation of China (NSFC) Grant No. 62276258.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ren, MS., Zhang, YM., Wang, QF., Yin, F., Liu, CL. (2024). Diff-Writer: A Diffusion Model-Based Stylized Online Handwritten Chinese Character Generator. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1964. Springer, Singapore. https://doi.org/10.1007/978-981-99-8141-0_7
Download citation
DOI: https://doi.org/10.1007/978-981-99-8141-0_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8140-3
Online ISBN: 978-981-99-8141-0
eBook Packages: Computer ScienceComputer Science (R0)