Recognizing online handwritten Chinese characters using RNNs with new computing architectures
Introduction
Intelligent electronic devices, such as mobile phone and pad, allow users to input characters in a handwriting way and are now widely used in our daily life. Handwriting is an easy way for people to interact with machines. Online handwritten Chinese character recognition (HCCR) has been studied for a long time and obtained great achievements during the past decades. However, it is still a challenging task to make the human-computer interaction way more smooth and natural.
There exist two types of handwriting, writing on the touchpad and gesture-based handwriting. The writing way on a touchpad is widely applied in many intelligent electronic devices. To make people naturally interact with computers, the gesture-based handwriting emerged in recent years. Compared with the conventional handwritten characters (writing on touch devices), the gesture-based handwritten characters are written in-air. Fig. 1(a) and (b) show the gesture-based writing system that we establish. Due to the different writing ways, there is a big difference between the two kinds of characters. A Chinese character generally consists of many strokes, however, the character always corresponds to only one stroke when it is written in the air. It means that there are not any stroke marks in the collected data. Additionally, characters are generated with more irregular character shapes or writing trajectories for the flexible motion of finger in the air without any constraint during writing. Fig. 1(c) and (d) show some examples to illustrate the difference between conventional handwritten characters and gesture-based handwritten characters, where we can see that the gesture-based handwritten characters have more irregular shapes [1].
Recognizing handwritten characters accurately and efficiently are very important for intelligent devices. Additionally, the Chinese written language has a large number of character classes. Most Chinese characters have a complex structure which makes the recognition task more difficult. Therefore, online handwritten Chinese character recognition (HCCR) is still a challenging task for researchers. With the success of deep learning [2], [3], the convolutional neural network (CNN) and recurrent neural network (RNN) have been successfully applied for online HCCR [4], [5], [6]. Of all the existing deep learning methods, the RNN is an excellent tool for sequence processing. Since one handwritten character is represented as a sequence of locations, it is very appropriate to use RNN for modeling handwritten characters [1], [5]. Compared with CNN, the RNN model directly deals with the raw sequential data and therefore has the potential to exploit information which is discarded in the image representations.
In this work, we propose a new RNN based end-to-end recognizer for online HCCR. As is known to all, parameter model size and computing efficiency are the two important indicators for evaluating a neural network system, So we expect to build a new system with higher computing efficiency, fewer parameters, and higher recognition accuracy, compared with the existing works. Motivated by that, we propose two new computing architectures:
- •
First, the variance constraint is used to increase the recognition accuracy of the system by decreasing the variances of hidden layer vectors.
- •
Second, the attention weight vector is introduced to describe the importance of hidden layer states at different time steps.
Section snippets
Related works
Online handwritten Chinese character recognition aims at recognizing characters accurately and efficiently. For the task of automatic recognition of handwritten Chinese characters, there are two main categories: conventional online HCCR and gesture-based online HCCR. We review the related works of them.
Basic RNN system
The basic architecture of RNN in the proposed system is shown in Fig. 2, which consists of one input layer, N hidden layers, and a fully-connected layer. In the application of gesture-based online HCCR, the writing trajectory of a character is represented as a sequence of dot locations where T denotes the sequence length of the character. The RNN receives an input of location and then the state of nth layer at time step t () is computed out through the
New computing architectures for RNN
Generally, more parameters can bring the increase of network performance. However, more parameters result in parameter redundancy and extra calculation burden. In this section, we introduce two new computing architectures for traditional RNN. We expect the RNN with new computing architectures to achieve better performance compared with the traditional RNN and involve fewer parameters.
Experimental results
The experiments are carried out on two handwritten Chinese character datasets, IAHCC-UCAS2016 dataset and ICDAR-2013 competition database.
The IAHCC-UCAS2016 dataset is a gesture-based handwritten Chinese characters dataset written by 115 writers, which contains 3873 character classes, i.e., 3811 Chinese characters, 52 case-sensitive English letters and 10 digits. For each class, we choose 93 samples as the training set, and the remaining 22 samples are used as testing set.
The dataset ICDAR-2013
Conclusion
This paper presents an end-to-end recognizer for online handwritten Chinese characters. In our system, two new computing architectures are proposed: (1) variance constraint and (2) attention weight vector. The variance constraint mechanism can effectively constrain the number of key parameters used for representing a single sample so that the number of samples that a certain parameter participates in computing decreases. It is beneficial for the parameters in RNN systems to more probably obtain
Acknowledgments
This work is supported by National Key R&D Program of China under contract No. 2017YFB1002203, NSFC projects under Grant 61772495, and NSFC Key Projects of International (Regional) Cooperation and Exchanges under Grant 61860206004.
Haiqing Ren is a Ph.D. student of University of Chinese Academy of Sciences. She obtained her B.E. degree from Yanshan University in 2012 and M.E. degree from Harbin Institute of Technology (HIT) in 2014. Her research interests include machine learning, pattern recognition, image processing and computer vision.
References (38)
- et al.
On-line handwriting character recognition using direction-change features that consider imaginary strokes
Pattern Recognit.
(1999) - et al.
Online and offline handwritten chinese character recognition: benchmarking on new databases
Pattern Recognit.
(2013) - et al.
In-air handwritten chinese character recognition with locality-sensitive sparse representation toward optimized prototype classifier
Pattern Recognit.
(2018) - et al.
Data augmentation and directional feature maps extraction for in-air handwritten chinese character recognition based on convolutional neural network
Pattern Recognit. Lett.
(2018) - et al.
An end-to-end recognizer for in-air handwritten chinese characters based on a new recurrent neural networks
(ICME, 2017) - et al.
Reducing the dimensionality of data with neural networks
Science
(2006) - et al.
Deep learning
Nature
(2015) Sparse arrays of signatures for online character recognition
Comput. Sci.
(2013)- et al.
Drawing and recognizing chinese characters with recurrent neural network
TPAMI
(2017) - et al.
Improved deep convolutional neural network for online handwritten chinese character recognition using domain-specific knowledge
ICDAR
(2015)
Online recognition of chinese characters: the state-of-the-art
TPAMI
A study on the use of 8-directional features for online handwritten chinese character recognition
ICDAR
Scut-couch2009-a comprehensive online unconstrained chinese handwriting database and benchmark evaluation
ICDAR
Imagenet classification with deep convolutional neural networks
NIPS
Learning a deep compact image representation for visual tracking
NIPS
A fast learning algorithm for deep belief nets
Neural Comput.
Multi-column deep neural networks for offline handwritten chinese character classification
IJCNN
Handwritten character recognition by alternately trained relaxation convolutional neural network
ICFHR
High performance offline handwritten chinese character recognition using googlenet and directional feature maps
ICDAR
Cited by (0)
Haiqing Ren is a Ph.D. student of University of Chinese Academy of Sciences. She obtained her B.E. degree from Yanshan University in 2012 and M.E. degree from Harbin Institute of Technology (HIT) in 2014. Her research interests include machine learning, pattern recognition, image processing and computer vision.
Weiqiang Wang received the B.E. and M.E. degrees in computer science from Harbin Engineering University, Harbin, China, in 1995 and 1998, respectively, and the Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China, in 2001. He is currently a Professor with the School of Computer and Controlling Engineering, University of CAS. His research interests include multimedia content analysis and computer vision.
Chenglin Liu received the B.S. degree in electronic engineering from Wuhan University, Wuhan, China; the M.E. degree in electronic engineering from Beijing University of Technology, Beijing, China; and the Ph.D. degree in pattern recognition and intelligent systems from the Institute of Automation of Chinese Academy of Sciences, Beijing, China, in 1989, 1992, and 1995, respectively. He was a Post-Doctoral Fellow with Korea Advanced Institute of Science and Technology and later with Tokyo University of Agriculture and Technology from 1996 to 1999. From 1999 to 2004, he was a Research Staff Member and later a Senior Researcher with the Central Research Laboratory, Hitachi, Ltd., Tokyo, Japan. Since 2005, he has been a Professor with the National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China, where he is currently the Director. His research interests include pattern recognition, image processing, neural networks, machine learning, and the applications to character recognition and document analysis. He has authored over 200 technical papers at prestigious international journals and conferences. He is a Fellow of the IAPR.