Abstract
This work aims to construct an end-to-end system that can accurately analyze the layout of a mathematical formula. To accomplish this, we proposed a deep-learning architecture that makes use of a Convolutional Graph Neural Network (ConvGNN)—a variant of a convolutional neural network that operates directly on graphs to label symbols and their spatial relationships within the context of mathematical formulas, while also factoring in the features of the neighbors and their interrelationships. Testing different models of ConvGNN and comparing with the classical method: a Multi Layers Perceptron (MLP), we confirm the effectiveness of ConvGNN, especially the spectral-based one, as a suitable way to model formula structure that a graph can represent. We demonstrate that our system competes with some related works in several experiments on CROHME, a handwritten math formula standard database. These tests revealed an accuracy of 82% for MLP, 23% for simple ConvGNN, and 92% for spectral-based ConvGNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Dimitriadis, Y.A., Coronado, J.L.: Towards an ART based mathematical editor, that uses online handwritten symbol recognition. Pattern Recogn. 28(6), 807–822 (1995)
Lee, M.C.: Understanding mathematical expressions using procedure-oriented transformation. Pattern Recogn. 27(3), 447–457 (1994)
Zonghan, W., Shirui, P., Zhang, C.: A comprehensive survey on graph neural networks. arXiv:1901.00596v4. Accessed 4 Dec 2019
Mouchère, H.: ICDAR 2019 competition on recognition of handwritten mathematical expressions and typeset formula detection. In: International Conference on Document Analysis and Recognition (ICDAR) (2019)
Vaz, T., Vernekar, N.: A survey on evaluating handwritten iterative mathematical expressions. Int. J. Res. Appl. Sci. Eng. Technol. 6(IV) (2018)
Padmapriya, R., Karpagavalli, S.: Offline handwritten mathematical expression recognition. Int. J. Innov. Res. Comput. Commun. Eng. 4(1), 52–59 (2016)
Gharde, S.S., Baviskar, P.V., Adhiya, K.P.: Identification of handwritten simple mathematical equation based on SVM and projection histogram. Int. J. Soft Comput. Eng. 3(2), 425–429 (2013). ISSN 2231-2307
Bharambe, M.: Recognition of offline handwritten mathematical expressions. Int. J. Comput. Appl. 2015, 35–39 (2015)
Shinde, S., Waghulade, R.: Handwritten mathematical expressions recognition using back propagation artificial neural network. Commun. Appl. Electron. 4(7), 1–6 (2016)
Jimenez, N., Nguyen, L.: Recognition of handwritten mathematical symbols with PHOG features (2013). http://cs229.stanford.edu
Fathima, I., Ashoka, K.: Machine learning approach for recognition of mathematical symbols. Int. J. Sci. Res. Eng. Technol. 6(8), 2278–0882 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ayeb, K.K., Echi, A.K. (2024). Off-Line Handwritten Math Formula Layout Analysis by ConvGNN. In: Bennour, A., Bouridane, A., Chaari, L. (eds) Intelligent Systems and Pattern Recognition. ISPR 2023. Communications in Computer and Information Science, vol 1940. Springer, Cham. https://doi.org/10.1007/978-3-031-46335-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-46335-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46334-1
Online ISBN: 978-3-031-46335-8
eBook Packages: Computer ScienceComputer Science (R0)