Abstract
Real scientific challenge, handwritten math formula recognition is an attractive field of pattern recognition leading to practical applications. Hundreds of alphanumeric and math symbols need to be recognized, many are so similar in appearance that some use of context is necessary for disambiguation. Analysis of the spatial relationships between symbols is challenging. In this work, we focus on handwritten math symbols and propose to recognize them by a deep learning approach. The symbol images, used for train, validation, and test are generated from Competition on Recognition of Online Handwritten Mathematical Expressions dataset (CROHME) 2019’s online patterns of mathematical symbols. As the large dataset is crucial for the performance of the deep learning model and it is labor-intensive to obtain a large amount of labeled data in real applications, we first augmented the database. Standing on the transfer learning technique, we then tested and compared several pre-trained Convolutional Neural networks (CNNs) like VGGNet, SqueezeNet, DenseNet, and Xception network and we tuned them to better fit our data. An accurate classification of 91.88% (train), 88.82% (validation), and 83.68% (test) for 101 classes is achieved, using only off-line features of the symbols.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ramadhan, I., Purnama, B., Al Faraby, S.: Convolutional neural networks applied to handwritten mathematical symbols classification. In: Fourth International Conference on Information and Communication Technologies (ICoICT) (2016)
Dong, L., Liu, H.: Recognition of offline handwritten mathematical symbols using convolutional neural networks. In: Zhao, Y., Kong, X., Taubman, D. (eds.) ICIG 2017. LNCS, vol. 10666, pp. 149–161. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71607-7_14
Nazemi, A., Tavakolian, N., Fitzpatrick, D., Fernando, C., Suen, C.Y.: Offline handwritten mathematical symbol recognition utilising deep learning, cs.CV (2019)
Phan, K.M., Le, A.D., Indurkhya, B., Nakagawa, M.: Augmented incremental recognition of online handwritten mathematical expressions. Int. J. Doc. Anal. Recogn. (IJDAR) 21(4), 253–268 (2018). https://doi.org/10.1007/s10032-018-0306-1
Julca-Aguilar, F.D., Hirata, N.S.T.: Symbol detection in online handwritten graphics using Faster R-CNN. In: International Workshop on Document Analysis Systems (DAS) (2018)
Mouchere, H., Viard-Gaudin, C., Zanibbi, R., Garain, U.: ICFHR2016 CROHME: competition on recognition of online handwritten mathematical expressions (2016)
Awal, A.M., Feng, G., Mouchere, H., Viard-Gaudin, C.: First experiments on a new online handwritten flowchart database. In: Document Recognition and Retrieval XVIII (2011)
MacLean, S., Labahn, G., Lank, E., Marzouk, M., Tausky, D.: Grammar-based techniques for creating ground-truthed sketch corpora. Int. J. Doc. Anal. Recogn. 14, 65–74 (2011). https://doi.org/10.1007/s10032-010-0118-4
Alvaro, F., Sanchez, J.A., Benedi, J.M.: Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recogn. Lett. 35, 58–67 (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional neural networks for large-scale image recognition. In: ICLR (2015)
Chollet, F.: Xception: deep learning with depthwise separable convolutions, cs.CV (2017)
Huang, G., Liu, Z., Maaten, L., Weinberger, K.: Densely connected convolutional networks, cs.CV (2018)
Rawat, W., Wang, Z.: Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 29(9), 2352–2449 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ayeb, K.K., Meguebli, Y., Echi, A.K. (2021). Deep Learning Architecture for Off-Line Recognition of Handwritten Math Symbols. In: Djeddi, C., Kessentini, Y., Siddiqi, I., Jmaiel, M. (eds) Pattern Recognition and Artificial Intelligence. MedPRAI 2020. Communications in Computer and Information Science, vol 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-71804-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-71804-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71803-9
Online ISBN: 978-3-030-71804-6
eBook Packages: Computer ScienceComputer Science (R0)