Abstract
Handwritten Chinese Essay Recognition (HCER) is a special branch of handwritten Chinese text recognition with great interest. In a naive way, it can be firstly segmented into text lines or even characters, followed by a text line or character recognition step. Instead, we propose an end-to-end recognition model named FPRNet which directly runs on full-page images in light of the segmentation-free strategy. Our well-designed model can extract text from a full-page image only supervised with text labels and adapt better to authentic noisy images. Besides, we propose an effective dimensionality reduction mechanism based on reshape operation to bridge features between 2D and 1D without information loss. Moreover, we propose an order-align strategy to mitigate the decoding confusion caused by skewness. Experiments are conducted on real-world essay images. Our model achieves a 5.83% character error rate (CER), which is comparable with the state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Su, T., Zhang, T., Guan, D., Huang, H.: Off-line recognition of realistic Chinese handwriting using segmentation-free strategy. Pattern Recogn. 42(1), 167–182 (2009)
Wang, Q., Yin, F., Liu, C.: Handwritten Chinese text recognition by integrating multiple contexts. IEEE TPAMI 34(8), 1469–1481 (2011)
Toledo, J., et al.: Handwriting recognition by attribute embedding and recurrent neural networks. In: ICDAR (2017)
Lei, K., et al.: Convolve, attend and spell: an attention-based sequence-to-sequence model for handwritten word recognition. In: German Conference on Pattern Recognition (2018)
Wu, Y., Yin, F., Liu, C.: Improving handwritten Chinese text recognition using neural network language models and convolutional neural network shape models. Pattern Recogn. 65, 251–264 (2017)
Graves, A., Fernández, S., Gomez, F., et al.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 369–376 (2006)
Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: CVPR (2018)
Chung, J., Delteil, T.: A computationally efficient pipeline approach to full page offline handwritten text recognition. In: ICDAR Workshop (2019)
Carbonell, M., Fornés, A., Villegas, M., et al.: A neural model for text localization, transcription and named entity recognition in full pages. Pattern Recogn. Lett. 136, 219–227 (2020)
Messina, R., Louradour, J.: Segmentation-free handwritten Chinese text recognition with LSTM-RNN. In: ICDAR (2015)
Wu, Y., Yin, F., et al.: Handwritten Chinese text recognition using separable multi-dimensional recurrent neural network. In: ICDAR (2017)
Yousef, M., Hussain, K.F., Mohammed, U.S.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recogn. 108, 107482 (2020)
Bluche, T.: Joint line segmentation and transcription for end-to-end handwritten paragraph recognition. In: NeuIPS (2016)
Wigington, C., Tensmeyer, C., Davis, B., et al.: Start, follow, read: end-to-end full-page handwriting recognition. In: ECCV (2018)
Coquenet, D., Chatelain, C., Paquet, T.: End-to-end handwritten paragraph text recognition using a vertical attention network. IEEE Trans. Pattern Anal. Mach. Intell. (2022). (Accepted)
Yousef, M., Bishop, T.: OrigamiNet: weakly-supervised, segmentation-free, one-step, full page text recognition by learning to unfold. In: CVPR (2020)
Howard, A., Zhu, M., Chen, B., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. ArXiv:1704.04861 (2017)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: CVPR (2017)
Wang, F., Jiang, M., Qian, C., et al.: Residual attention network for image classification. In: CVPR (2017)
Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway networks. ArXiv:1505.00387 (2015)
Coquenet, D., Chatelain, C., Paquet, T.: Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network. In: International Conference on Frontiers in Handwriting Recognition, pp. 19–24 (2020)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: CVPR (2015)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. ArXiv:1412.6980 (2014)
Yousef, M., Hussain, K., Mohammed, U.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recogn. 108, 107482 (2020)
Coquenet, D., Chatelain, C., Paquet, T.: SPAN: a simple predict and align network for handwritten paragraph recognition. In: International Conference on Document Analysis and Recognition (2021)
Acknowledgment
Great thanks to Jifeng Wang. This work was supported by the National Key Research and Development Program of China (Grant No. 2020AAA0108003) and National Natural Science Foundation of China (Grant No. 62277011 and 61673140).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Su, T., You, H., Liu, S., Wang, Z. (2022). FPRNet: End-to-End Full-Page Recognition Model for Handwritten Chinese Essay. In: Porwal, U., Fornés, A., Shafait, F. (eds) Frontiers in Handwriting Recognition. ICFHR 2022. Lecture Notes in Computer Science, vol 13639. Springer, Cham. https://doi.org/10.1007/978-3-031-21648-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-21648-0_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21647-3
Online ISBN: 978-3-031-21648-0
eBook Packages: Computer ScienceComputer Science (R0)