Abstract
Handwritten mathematical expression and symbol recognition is a subfield of document image analysis that aims to convert images of handwritten mathematical formulas into a machine-readable format. Despite decades of research, recognition of handwritten mathematical expressions and symbols, particularly those written in Arabic, remains a challenging problem. To address this issue, we propose a Deep Neural Network (DNN)-based approach for Handwritten Arabic Mathematical Symbol Recognition. DNNs are powerful tools for data analysis and image classification, but they are susceptible to overfitting. Additionally, there are limited large-scale databases available for Arabic mathematical symbol recognition. To overcome this, we propose using MixUp data augmentation to increase the diversity of available data. MixUp involves training a CNN on convex combinations of pairs of examples and their corresponding labels. Furthermore, we integrate our new framework with both Cross Entropy loss and triplet loss on the augmented samples, which significantly improves the classification accuracy. The experimental results conducted on the Handwritten Arabic Mathematical Dataset (HAMF) [1] demonstrate that our proposed framework yields a significant improvement in accuracy compared to the standard CNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hadj Ali, I., Mahjoub, M.A.: Database of handwritten Arabic mathematical formula images. In: 2016 13th International Conference on Computer Graphics, Imaging and Visualization (CGiV), pp. 145–149. IEEE (2016)
Li, Z., Wang, X., Liu, Y., et al.: Improving handwritten mathematical expression recognition via similar symbol distinguishing. IEEE Trans. Multimed. (2023)
Zhang, J., Du, J., Zhang, S., et al.: Watch, attend and parse: an end-to-end neural network-based approach to handwritten mathematical expression recognition. Pattern Recogn. 71, 196–206 (2017)
Alvaro, F., Sánchez, J.-A., Benedí, J.-M.: Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recogn. Lett. 35, 58–67 (2014)
Zhang, H., Cisse, M., Dauphin, Y.N., et al.: MixUp: beyond empirical risk minimization. arXiv:1710.09412 (2017)
Zanibbi, R., Blostein, D., Cordy, J.R.: Recognizing mathematical expressions using tree transformation. IEEE Trans. Pattern Anal. Mach. Intell. 24(11), 1455–1467 (2002)
Hadj Ali, I., Mahjoub, M.A.: Structure relationship classification for the recognition of mathematical expression handwritten in Arabic. In: 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–6. IEEE (2020)
Yuan, Y., Liu, X., Dikubab, W., et al.: Syntax-aware network for handwritten mathematical expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4553–4562 (2022)
Anderson, R.H.: Syntax-directed recognition of hand-printed two-dimensional mathematics. In: Symposium on Interactive Systems for Experimental Applied Mathematics, pp. 436–459. Association for Computing Machinery Inc. (1967)
Awal, A.-M., Mouchère, H., Viard-Gaudin, C.: Towards handwritten mathematical expression recognition. In: 2009 10th International Conference on Document Analysis and Recognition, pp. 1046–1050. IEEE (2009)
Zhang, J., Du, J., Dai, L.: Multi-scale attention with dense encoder for handwritten mathematical expression recognition. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2245–2250. IEEE (2018)
Ali, I.H., Mahjoub, M.A.: Dynamic random forest for the recognition of Arabic handwritten mathematical symbols with a novel set of features. Int. Arab J. Inf. Technol. 15(3A), 565–575 (2018)
Álvaro, F., Sánchez, J.A.: Comparing several techniques for offline recognition of printed mathematical symbols. In: 2010 20th International Conference on Pattern Recognition, pp. 1953–1956. IEEE (2010)
Hirata, N.S.T., Honda, W.Y.: Automatic labeling of handwritten mathematical symbols via expression matching. In: Jiang, X., Ferrer, M., Torsello, A. (eds.) GbRPR 2011. LNCS, vol. 6658, pp. 295–304. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20844-7_30
Chan, K.-F., Yeung, D.-Y.: Elastic structural matching for online handwritten alphanumeric character recognition. In: Proceedings of Fourteenth International Conference on Pattern Recognition (Cat. No. 98EX170), pp. 1508–1511. IEEE (1998)
Keshari, B., Watt, S.: Hybrid mathematical symbol recognition using support vector machines. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), pp. 859–863. IEEE (2007)
Hu, L., Zanibbi, R.: HMM-based recognition of online handwritten mathematical symbols using segmental k-means initialization and a modified pen-up/down feature. In: 2011 International Conference on Document Analysis and Recognition, pp. 457–462. IEEE (2011)
Ali, I.H., Mahjoub, M.A.: Random forests for the recognition of handwritten Arabic mathematical symbols (2017)
Le, A.D., Indurkhya, B., Nakagawa, M.: Pattern generation strategies for improving recognition of handwritten mathematical expressions. Pattern Recogn. Lett. 128, 255–262 (2019)
Shams, M., Elsonbaty, A., Elsawy, W., et al.: Arabic handwritten character recognition based on convolution neural networks and support vector machine. arXiv preprint arXiv:2009.13450 (2020)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Sabri, M., Kurita, T.: Effect of additive noise for multi-layered perceptron with autoencoders. IEICE Trans. Inf. Syst. 100(7), 1494–1504 (2017)
Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24261-3_7
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)
Thulasidasan, S., Chennupati, G., Bilmes, J.A., et al.: On mixup training: improved calibration and predictive uncertainty for deep neural networks. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Huang, G., Liu, Z., van der Maaten, L., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp. 448–456. PMLR (2015)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, pp. 315–323 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hadj Ali, I., Ben Khalifa, A., Mahjoub, M.A. (2024). MixUp Data Augmentation for Handwritten Arabic Mathematical Symbols Recognition. In: Mosbah, M., et al. Advances in Model and Data Engineering in the Digitalization Era. MEDI 2023. Communications in Computer and Information Science, vol 2071. Springer, Cham. https://doi.org/10.1007/978-3-031-55729-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-55729-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55728-6
Online ISBN: 978-3-031-55729-3
eBook Packages: Computer ScienceComputer Science (R0)