Abstract
Recently, reliable quality images can be generated due to the development of adversarial generative networks. Nevertheless, computer-generated images are still not comparable to humans in terms of handwriting image generation. In this paper, a novel method (i.e. AFFGANwriting) based on multi-scale features fusion has been proposed for handwriting image generation. In AFFGANwriting, a style encoder based on VGG19 has been designed to extract features in different scales. In this way, a variety of global features (e.g. stroke thickness, inclination and so on) and local features (e.g. continuous strokes, personalized writing and so forth) can be obtained. After that, the global features and the local features can be fused together to generate much more realistic handwriting images by multiple feature fusion modules of AFFGANwriting. Experimental results demonstrate that the proposed method can be competent for the task of handwriting images generation and outperforms various baseline and state-of-the-art methods. The code is available at: https://github.com/wh807088026/AFFGanWriting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kang, L., Riba, P., Wang, Y., Rusiñol, M., Fornés, A., Villegas, M.: GANwriting: content-conditioned generation of styled handwritten word images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 273–289. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_17
Dai, Y., Gieseke, F., Oehmcke, S., et al.: Attentional feature fusion. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3560–3569 (2021)
Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
Aksan, E., Pece, F., Hilliges, O.: DeepWriting: Making digital ink editable via deep generative modeling. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2018)
Aksan, E., Hilliges, O.: STCN: stochastic temporal convolutional networks. arXiv preprint arXiv:1902.06568, 2019
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial networks. Commun. ACM 63, 139–144 (2020)
Zhu, J.Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Alonso, E., Moysset, B., Messina, R.: Adversarial generation of handwritten text images conditioned on sequences. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 481–486. IEEE (2019)
Fogel, S., Averbuch-Elor, H., Cohen, S., et al.: Scrabblegan: semi-supervised varying length handwritten text generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4324–4333 (2020)
Davis, B., Tensmeyer, C., Price, B., et al.: Text and style conditioned GAN for generation of offline handwriting lines. arXiv preprint arXiv:2009.00678 (2020)
Mattick, A., Mayr, M., Seuret, M., Maier, A., Christlein, V.: SmartPatch: improving handwritten word imitation with patch discriminators. In: Lladós, J., Lopresti, D., Uchida, S. (eds.) ICDAR 2021. LNCS, vol. 12821, pp. 268–283. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86549-8_18
Bhunia, A.K., Khan, S., Cholakkal, H., et al.: Handwriting transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1086–1094 (2021)
Gan, J., Wang, W.: HiGAN: handwriting imitation conditioned on arbitrary-length texts and disentangled styles. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 7484–7492 (2021)
Kang, L., Riba, P., Rusinol, M., et al.: Content and style aware generation of text-line images for handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 8846–8860 (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5, 39–46 (2002)
Heusel, M., Ramsauer, H., Unterthiner, T., et al.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017)
Wang, Y., Wang, H., Sun, S., et al.: An approach based on transformer and deformable convolution for realistic handwriting samples generation. In: 2022 26th International Conference on Pattern Recognition (ICPR), pp. 1457–1463. IEEE (2022)
Acknowledgment
This study is supported by the Project for Science and Technology of Inner Mongolia Autonomous Region under Grant 2019GG281, the Natural Science Foundation of Inner Mongolia Autonomous Region under Grant 2019ZD14, and the Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region under Grant NJYT-20-A05.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, H., Wang, Y., Wei, H. (2023). AFFGANwriting: A Handwriting Image Generation Method Based on Multi-feature Fusion. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14190. Springer, Cham. https://doi.org/10.1007/978-3-031-41685-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-41685-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41684-2
Online ISBN: 978-3-031-41685-9
eBook Packages: Computer ScienceComputer Science (R0)
