Skip to main content
SpringerLink
Log in

Intelligent Chinese calligraphy beautification from handwritten characters for robotic writing

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Chinese calligraphy is the artistic expression of character writing and is highly valued in East Asia. However, it is a challenge for non-expert users to write visually pleasing calligraphy with his or her own unique style. In this paper, we develop an intelligent system that beautifies Chinese handwriting characters and physically writes them in a certain calligraphy style using a robotic arm. First, we sketch the handwriting characters using a mouse or a touch pad. Then, we employ a convolutional neural network to identify each stroke from the skeletons, and the corresponding standard stroke is retrieved from a pre-built calligraphy stroke library for robotic arm writing. To output aesthetically beautiful calligraphy with the user’s style, we propose a global optimization approach to solve the minimization problem between the handwritten strokes and standard calligraphy strokes, in which a shape character vector is presented to describe the shape of standard strokes. Unlike existing systems that focus on the generation of digital calligraphy from handwritten characters, our system has the advantage of converting the user-input handwriting into physical calligraphy written by a robotic arm. We take the regular script (Kai) style as an example and perform a user study to evaluate the effectiveness of the system. The writing results show that our system can achieve visually pleasing calligraphy from various input handwriting while retaining the user’s style.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Chao, F., Lv, J., Zhou, D., Yang, L., Lin, C., Shang, C., Zhou, C.: Generative adversarial nets in robotic Chinese calligraphy. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1104–1110 (2018)

  2. Chen, X., Lian, Z., Tang, Y., Xiao, J.: A benchmark for stroke extraction of Chinese character. Acta Sci. Nat. Univ. Pekin. 52(1), 49–57 (2016)

    MathSciNet  Google Scholar 

  3. Dobot Robotic Arms: https://www.dobot.cc/. Accessed on Feb 10, 2019

  4. Furrer, F., Wermelinger, M., Yoshida, H., Gramazio, F., Kohler, M., Siegwart, R., Hutter, M.: Autonomous robotic stone stacking with online next best object target pose planning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2350–2356 (2017)

  5. Gupta, A., Srivastava, M., Mahanta, C.: Offline handwritten character recognition using neural network. In: 2011 IEEE International Conference on Computer Applications and Industrial Electronics (ICCAIE), pp. 102–107 (2011)

  6. Hashiguchi, H., Arimoto, S., Ozawa, R.: Control of a handwriting robot with DOF-redundancy based on feedback in task-coordinates. J. Robot. Mechatron. 16, 381–387 (2004)

    Article  Google Scholar 

  7. Igarashi, T., Inami, M.: Exploration of alternative interaction techniques for robotic systems. IEEE Comput. Graph. Appl. 35(3), 33–41 (2015)

    Article  Google Scholar 

  8. Igarashi, T., Matsuoka, S., Tanaka, H.: Teddy: A sketching interface for 3D freeform design. In: SIGGRAPH’99, pp. 409–416, New York (1999)

  9. Kim, B., Wang, O., Öztireli, A.C., Gross, M.: Semantic segmentation for line drawing vectorization using neural networks. Comput. Graph. Forum 37(2), 329–338 (2018)

    Article  Google Scholar 

  10. Kim, S., Jo, J., Oh, Y., Oh, S., Srinivasa, S., Likhachev, M.: Robotic handwriting: multi-contact manipulation based on reactional internal contact hypothesis. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 877–884 (2014)

  11. Krizek, M., Neittaanmki, P., Glowinski, R., Korotov, S.: Conjugate Gradient Algorithms and Finite Element Methods. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  12. Li, H., Liu, P., Xu, S., Lin, S.: Calligraphy beautification method for Chinese handwritings. In: 2012 4th International Conference on Digital Home, pp. 122–127 (2012)

  13. Liu, L., Xia, W., Jin, L., Mao, H., Tian, F.: A Kai style contour beautification method for Chinese handwriting characters. In: 2010 IEEE International Conference on Systems, Man and Cybernetics, pp. 3644–3649 (2010)

  14. Lo, K.W., Kwok, K.W., Wong, S.M., Yam, Y.: Brush footprint acquisition and preliminary analysis for Chinese calligraphy using a robot drawing platform. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5183–5188 (2006)

  15. Ma, Z., Su, J.: Aesthetics evaluation for robotic Chinese calligraphy. IEEE Trans. Cogn. Dev. Syst. 9(1), 80–90 (2017)

    Article  Google Scholar 

  16. Mueller, S., Huebel, N., Waibel, M., D’Andrea, R.: Robotic calligraphy-learning how to write single strokes of Chinese and Japanese characters. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1734–1739 (2013)

  17. Okutomi, M.: Digital Image Processing. The Computer Graphic Arts Society (CG-ARTS), Tokyo (2015)

    Google Scholar 

  18. Online Chinese Calligraphy Generator: https://www.zhenhaotv.com/. Accessed on Dec. 4, 2018

  19. Sun, Y., Ding, N., Qian, H., Xu, Y.: A robot for classifying Chinese calligraphic types and styles. In: 2013 IEEE International Conference on Robotics and Automation, pp. 4279–4284 (2013)

  20. Sun, Y., Qian, H., Xu, Y.: A geometric approach to stroke extraction for the Chinese calligraphy robot. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3207–3212 (2014)

  21. Sun, Y., Qian, H., Xu, Y.: Robot learns Chinese calligraphy from demonstrations. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4408–4413. IEEE, New York (2014)

  22. Syamlan, A.T., Nurhadi, H., Pramujati, B.: Character recognition for writing robot control using ANFIS. In: 2015 International Conference on Advanced Mechatronics, Intelligent Manufacture, and Industrial Automation (ICAMIMIA), pp. 46–48 (2015)

  23. Tian, X., Tian, Y.: The Elaboration of Ancient and Modern Famous Kai Calligraphy. Jiangxi Fine Arts Press, Jiangxi (2007)

    Google Scholar 

  24. Xiao, X., Jin, L., Yang, Y., Yang, W., Sun, J., Chang, T.: Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition. Pattern Recognit. 72, 72–81 (2017)

    Article  Google Scholar 

  25. Yao, F., Shao, G., Yi, J.: Trajectory generation of the writing-brush for a robot arm to inherit block-style Chinese character calligraphy techniques. Adv. Robot. 18(3), 331–356 (2004)

    Article  Google Scholar 

  26. Yi, T., Lian, Z., Tang, Y., Xiao, J.: A data-driven personalized digital ink for Chinese characters. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MultiMedia Modeling, pp. 254–265. Springer, New York (2014). https://doi.org/10.1007/978-3-319-04114-8

  27. Zeng, H., Huang, Y., Chao, F., Zhou, C.: Survey of robotic calligraphy research. CAAI Trans. Intell. Syst. 11(1), 15–26 (2016)

    Google Scholar 

  28. Zhang, Z., Wu, J., Yu, K.: Chinese calligraphy specific style rendering system. In: Proceedings of the ACM International Conference on Digital Libraries, pp. 99–108 (2010)

Download references

Acknowledgements

We thank the CGI2019 reviewers for their thoughtful comments. The work is supported by the NSFC (61303124), NSBR Plan of Shaanxi (2019JM370) and the Fundamental Research Funds for the Central Universities (2452017343).

Author information

Authors and Affiliations

  1. College of Information Engineering, Northwest A&F University, Yangling, China

    Xinyue Zhang, Yuanhao Li, Zhiyi Zhang & Shaojun Hu

  2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture, Yangling, China

    Zhiyi Zhang & Shaojun Hu

  3. Faculty of Science and Engineering, Iwate University, Morioka, Japan

    Kouichi Konno

Authors
  1. Xinyue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kouichi Konno
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaojun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaojun Hu.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 22575 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Li, Y., Zhang, Z. et al. Intelligent Chinese calligraphy beautification from handwritten characters for robotic writing. Vis Comput 35, 1193–1205 (2019). https://doi.org/10.1007/s00371-019-01675-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01675-w

Keywords

Search

Navigation