Skip to main content
SpringerLink
Log in

Improved Learning for Online Handwritten Chinese Text Recognition with Convolutional Prototype Network

  • Conference paper
  • First Online:
Document Analysis and Recognition - ICDAR 2023 (ICDAR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14190))

Included in the following conference series:

Abstract

Segmentation-based handwritten text recognition has the advantage of character interpretability but needs a character classifier with high classification accuracy and non-character rejection capability. The classifier can be trained on both character samples and string samples but real string samples are usually insufficient. In this paper, we proposed a learning method for segmentation-based online handwritten Chinese text recognition with a convolutional prototype network as the underlying classifier. The prototype classifier is inherently resistant to non-characters, and so, can be trained with character and string samples without the need of data augmentation. The learning has two stages: pre-training on character samples with a modified loss function for improving non-character resistance, and weakly supervised learning on both character and string samples for improving recognition performance. Experimental results on the CASIA-OLHWDB and ICDAR2013-Online datasets show that the proposed method can achieve promising recognition performance without training data augmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu, C.-L., Jäger, S., Nakagawa, M.: Online recognition of Chinese characters: the state-of-the-art. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 198–213 (2004)

    Article  Google Scholar 

  2. Liu, C.-L., Nakagawa, M.: Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition. Pattern Recognit. 34(3), 601–615 (2001)

    Article  MATH  Google Scholar 

  3. Wang, D.-H., Liu, C.-L., Zhou, X.-D.: An approach for real-time recognition of online Chinese handwritten sentences. Pattern Recognit. 45(10), 3661–3675 (2012)

    Article  Google Scholar 

  4. Zhou, X.-D., Wang, D.-H., Tian, F., Liu, C.-L., Nakagawa, M.: Handwritten Chinese/Japanese text recognition using semi-Markov conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2413–2426 (2013)

    Article  Google Scholar 

  5. Wu, Y.-C., Fei, Y., Liu, C.-L.: Improving handwritten Chinese text recognition using neural network language models and convolutional neural network shape models. Pattern Recognit. 65, 251–264 (2017)

    Google Scholar 

  6. Wang, Z.-X., Wang, Q.-F., Yin, F., Liu, C.-L.: Weakly supervised learning for over-segmentation based handwritten Chinese text recognition. In: ICFHR 2020, pp. 157–162 (2020)

    Google Scholar 

  7. Su, T.-H., Zhang, T., Guan, D.-J., Huang, H.-J.: Off-line recognition of realistic Chinese handwriting using segmentation-free strategy. Pattern Recognit. 42(1), 167–182 (2009)

    Google Scholar 

  8. Jiang, Z.-W., Ding, X.-Q., Liu, C., Wang, Y.-W.: A novel short merged off-line handwritten Chinese character string segmentation algorithm using hidden Markov model. In: ICDAR 2011, pp. 668–672 (2011)

    Google Scholar 

  9. Jayech, K., Mahjoub, M.A., Amara, N.E.B.: Synchronous multi-stream hidden Markov model for offline Arabic handwriting recognition without explicit segmentation. Neurocomputing 214, 958–971 (2016)

    Article  Google Scholar 

  10. Peng, D.-Z., et al.: Recognition of handwritten Chinese text by segmentation: a segment-annotation-free approach. IEEE Trans. Multimed. (2022)

    Google Scholar 

  11. Wang, Z.-R., Jun, D., Wang, J.-M.: Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition. Pattern Recognit. 100, 107102 (2020)

    Article  Google Scholar 

  12. Liu, C.-L., Yin, F., Wang, D.-H., Wang, Q.-F.: Online and offline handwritten Chinese character recognition: benchmarking on new databases. Pattern Recognit. 46(1), 155–162 (2013)

    Article  Google Scholar 

  13. Yang, H.-M., Zhang, X.-Y., Yin, F., Yang, Q., Liu, C.-L.: Convolutional prototype network for open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 44(5), 2358–2370 (2022)

    Google Scholar 

  14. Liu, C.-L., Sako, H., Fujisawa, H.: Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1395–1407 (2004)

    Article  Google Scholar 

  15. Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: CVPR 2012, pp. 3642–3649

    Google Scholar 

  16. Zhang, X.-Y., Bengio, Y., Liu, C.-L.: Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark. Pattern Recognit. 61, 348–360 (2017)

    Article  Google Scholar 

  17. Liu, C.-L.: Normalization-cooperated gradient feature extraction for handwritten character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1465–1469 (2007)

    Article  Google Scholar 

  18. Liu, X., Hu, B.-T., Chen, Q.-C., Xiang-Ping, W., You, J.-H.: Stroke sequence-dependent deep convolutional neural network for online handwritten chinese character recognition. IEEE Trans. Neural Networks Learn. Syst. 31(11), 4637–4648 (2020)

    Google Scholar 

  19. Ren, H.-Q., Wang, W.-Q., Xi-Wen, Q., Cai, Y.-Q.: A new hybrid-parameter recurrent neural network for online handwritten Chinese character recognition. Pattern Recognit. Lett. 128, 400–406 (2019)

    Article  Google Scholar 

  20. Xie, Z.-C., Sun, Z.-H., Jin, L.-W., Ni, H., Lyons, T.J.: Learning spatial-semantic context with fully convolutional recurrent network for online handwritten Chinese text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(8), 1903–1917 (2018)

    Article  Google Scholar 

  21. Chen, K., et al.: A compact CNN-DBLSTM based character model for online handwritten Chinese text recognition. In: ICDAR 2017, pp. 1068–1073

    Google Scholar 

  22. Qin, X.-H., Zhang, H.-Y., Ke, X., Shen, Z.-H., Qi, S.-M., Liu, K.: Progressive multitask learning network for online Chinese signature segmentation and recognition. In: ICFHR 2022, pp. 153–167

    Google Scholar 

  23. Lee, S.-W., Song, H.-H.: Optimal design of reference models for large-set handwritten character recognition. Pattern Recognit. 27(9), 1267–1274 (1994)

    Article  Google Scholar 

  24. Niu, S.-C., et al.: Towards stable test-time adaptation in dynamic wild world. ICLR (2023)

    Google Scholar 

  25. Liu, C.-L., Kim, I.-J., Kim, J.H.: High accuracy handwritten Chinese character recognition by improved feature matching method. In: ICDAR 1997, pp. 1033–1037

    Google Scholar 

  26. Raghavendra, B.S., Narayanan, C.K., Sita, G., Ramakrishnan, A.G., Sriganesh, M.: Prototype learning methods for online handwriting recognition. In: ICDAR 2005, pp. 287–291

    Google Scholar 

  27. Liu, C.-L.: One-vs-all training of prototype classifier for pattern classification and retrieval. In: ICPR 2010, pp. 3328–3331

    Google Scholar 

  28. Impedovo, S., Mangini, F.M., Barbuzzi, D.: A novel prototype generation technique for handwriting digit recognition. Pattern Recognit. 47(3), 1002–1010 (2014)

    Article  Google Scholar 

  29. Ao, X., Zhang, X.-Y., Liu, C.-L.: Cross-modal prototype learning for zero-shot handwritten character recognition. Pattern Recognit. 131, 108859 (2022)

    Article  Google Scholar 

  30. Yang, H.-M., Zhang, X.-Y., Yin, F., Liu, C.-L.: Robust classification with convolutional prototype learning. In: CVPR 2018, pp. 3474–3482

    Google Scholar 

  31. Gao, L.-K., Zhang, H., Liu, C-L.: Handwritten text recognition with convolutional prototype network and most aligned frame based CTC training. ICDAR (1), pp. 205–220 (2021)

    Google Scholar 

  32. Yu, M.-M., Zhang, H., Yin, F., Liu, C.-L.: An efficient prototype-based model for handwritten text recognition with multi-loss fusion. In: ICFHR 2022, pp. 404–418

    Google Scholar 

  33. Heng Zhang, Cheng-Lin Liu: A Lattice-Based Method for Keyword Spotting in Online Chinese Handwriting. ICDAR 2011: 1064–1068

    Google Scholar 

  34. Liu, C.-L., Nakagawa, M.: Precise Candidate Selection for Large Character Set Recognition by Confidence Evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), 636–642 (2000)

    Article  Google Scholar 

  35. Jeffrey A. Barnett: Computational Methods for A Mathematical Theory of Evidence. Classic Works of the Dempster-Shafer Theory of Belief Functions 2008: 197–216

    Google Scholar 

  36. Liu, C.-L., Yin, F., Wang, D.-H., Wang, Q.-F.: CASIA online and offline Chinese handwriting databases. In: ICDAR 2011, pp. 37–41 (2011)

    Google Scholar 

  37. Yin, F., Wang, Q.-F., Zhang, X.-Y., Liu, C.-L.: ICDAR 2013 Chinese handwriting recognition competition. In: ICDAR 2013, pp. 1464–1470

    Google Scholar 

  38. Shi, B.-G., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2017)

    Article  Google Scholar 

  39. Zhou, X.-D., Zhang, Y.-M., Tian, F., Wang, H.-A., Liu, C.-L.: Minimum-risk training for semi-Markov conditional random fields with application to handwritten Chinese/Japanese text recognition. Pattern Recognit. 47(5), 1904–1916 (2014)

    Article  Google Scholar 

  40. Peng, D.-Z., Jin, L.-W., Wu, Y.-Q., Wang, Z.-P., Cai, M.-X.: A fast and accurate fully convolutional network for end-to-end handwritten Chinese text segmentation and recognition. In: ICDAR 2019, pp. 25–30

    Google Scholar 

Download references

Acknowledgement

This work has been supported by the National Key Research and Development Program Grant 2020AAA0109700, the National Natural Science Foundation of China (NSFC) grant 61936003.

Author information

Authors and Affiliations

  1. State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institution of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Yi Chen, Heng Zhang & Cheng-Lin Liu

  2. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China

    Yi Chen & Cheng-Lin Liu

Authors
  1. Yi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng-Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheng-Lin Liu .

Editor information

Editors and Affiliations

  1. TU Dortmund University, Dortmund, Germany

    Gernot A. Fink

  2. Adobe, College Park, MN, USA

    Rajiv Jain

  3. Osaka Metropolitan University, Osaka, Japan

    Koichi Kise

  4. Rochester Institute of Technology, Rochester, NY, USA

    Richard Zanibbi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Y., Zhang, H., Liu, CL. (2023). Improved Learning for Online Handwritten Chinese Text Recognition with Convolutional Prototype Network. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14190. Springer, Cham. https://doi.org/10.1007/978-3-031-41685-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-41685-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-41684-2

  • Online ISBN: 978-3-031-41685-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation