Skip to main content
Springer Nature Link
Log in

Overview of Mathematical Expression Recognition

  • Conference paper
  • First Online:
Pattern Recognition and Artificial Intelligence (ICPRAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12068))

  • 1594 Accesses

Abstract

Mathematical expression recognition has been one of the most fascinating research among the various researches in field of image processing. This problem typically consists of three major stages, namely, expression positioning, symbol segmentation, symbol recognition, and structural analysis. In this paper, we will review most of the existing work with respect to each of the major stages of the recognition process. Moreover, some important issues in mathematical expression recognition, like handwritten MEs, will be addressed in depth. Finally, point out the future research directions of mathematical expression.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson, R.H.: Syntax-directed recognition of hand-printed two-dimensional mathematics. In: Symposium on Interactive Systems for Experimental Applied Mathematics: Proceedings of the Association for Computing Machinery Inc. Symposium, pp. 436–459. ACM (1967)

    Google Scholar 

  2. Belaid, A., Haton, J.-P.: A syntactic approach for handwritten mathematical formula recognition. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(1), 105–111 (1984)

    Article  Google Scholar 

  3. Chan, K.-F., Yeung, D.-Y.: Mathematical expression recognition: a survey. Int. J. Doc. Anal. Recogn. 3(1), 3–15 (2000). https://doi.org/10.1007/PL00013549

    Article  MathSciNet  Google Scholar 

  4. Zanibbi, R., Blostein, D.: Recognition and retrieval of mathematical expressions. Int. J. Doc. Anal. Recogn. 15(4), 331–357 (2012)

    Article  Google Scholar 

  5. Ha, J., Haralick, R.M., Phillips, I.T.: Recursive X-Y cut using bounding boxes of connected components. In: International Conference on Document Analysis & Recognition. IEEE (1995)

    Google Scholar 

  6. Kumar, P.P., Agarwal, A., Bhagvati, C.: A rule-based approach to form mathematical symbols in printed mathematical expressions. In: Sombattheera, C., Agarwal, A., Udgata, S.K., Lavangnananda, K. (eds.) MIWAI 2011. LNCS (LNAI), vol. 7080, pp. 181–192. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25725-4_16

    Chapter  Google Scholar 

  7. Yoo, Y.H., Kim, J.H.: Mathematical formula recognition based on modified recursive projection profile cutting and labeling with double linked list. In: Kim, J.H., Matson, E., Myung, H., Xu, P. (eds.) Robot Intelligence Technology and Applications 2012. AISC, vol. 208, pp. 983–992. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37374-9_95

    Chapter  Google Scholar 

  8. Farulla, G.A., Armano, T., Capietto, A., Murru, N., Rossini, R.: Artificial Neural Networks and Fuzzy Logic for Recognizing Alphabet Characters and Mathematical Symbols. In: Miesenberger, K., Bühler, C., Penaz, P. (eds.) ICCHP 2016. LNCS, vol. 9758, pp. 7–14. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41264-1_1

    Chapter  Google Scholar 

  9. Hirata, N.S.T., Honda, W.Y.: Automatic labeling of handwritten mathematical symbols via expression matching. In: Jiang, X., Ferrer, M., Torsello, A. (eds.) GbRPR 2011. LNCS, vol. 6658, pp. 295–304. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20844-7_30

    Chapter  Google Scholar 

  10. MacLean, S., Labahn, G.: A new approach for recognizing handwritten mathematics using relational grammars and fuzzy sets. Int. J. Doc. Anal. Recogn. 16, 139–163 (2013). https://doi.org/10.1007/s10032-012-0184-x

    Article  Google Scholar 

  11. Maclean, S., Labahn, G.: A Bayesian model for recognizing handwritten mathematical expressions. Pattern Recogn. 48(8), 2433–2445 (2015)

    Article  Google Scholar 

  12. Simistira, F., Papavassiliou, V., Katsouros, V., et al.: A system for recognition of on-line handwritten mathematical expressions. In: International Conference on Frontiers in Handwriting Recognition. IEEE Computer Society (2012)

    Google Scholar 

  13. Álvaro, F., Sánchez, J.-A., Benedí, J.-M.: Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recogn. Lett. 35(1), 58–67 (2014)

    Article  Google Scholar 

  14. Awal, A.M., Mouchère, H., Viard-Gaudin, C.: A global learning approach for an online handwritten mathematical expression recognition system. Pattern Recogn. Lett. 35, 68–77 (2014)

    Article  Google Scholar 

  15. Hu, Y., Peng, L., Tang Y.: On-line handwritten mathematical expression recognition method based on statistical and semantic analysis. In: 2014 11th IAPR International Workshop on Document Analysis Systems (DAS). IEEE Computer Society (2014)

    Google Scholar 

  16. Le, A.D., Nakagawa, M.: Training an end-to-end system for handwritten mathematical expression recognition by generated patterns. In: ICDAR 2017. IEEE (2017)

    Google Scholar 

  17. Le, A.D., Nguyen, H.D., Indurkhya, B., Nakagawa, M.: Stroke order normalization for improving recognition of online handwritten mathematical expressions. Int. J. Doc. Anal. Recogn. 22(1), 29–39 (2019). https://doi.org/10.1007/s10032-019-00315-2

    Article  Google Scholar 

  18. Zhang, T., Mouchère, H., Viard-Gaudin, C.: A tree-BLSTM-based recognition system for online handwritten mathematical expressions. Neural Comput. Appl. 32(9), 4689–4708 (2018). https://doi.org/10.1007/s00521-018-3817-2

    Article  Google Scholar 

  19. Zhang, J., Jun, D., Lirong, D.: Track, Attend and Parse (TAP): an end-to-end framework for online handwritten mathematical expression recognition. IEEE Trans. Multimedia 21(1), 221–233 (2019)

    Article  Google Scholar 

  20. Zhang, J., Jun, D., Lirong, D.: A GRU-based encoder-decoder approach with attention for online handwritten mathematical expression recognition. In: International Conference on Document Analysis and Recognition, pp. 902–907 (2017)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the Guangdong Provincial Government of China through the “Computational Science Innovative Research Team” program and the Guangdong Province Key Laboratory of Computational Science at the Sun Yat-Sen University, and the National Science Foundation of China (grant no. 11471012).

Author information

Authors and Affiliations

  1. School of Mathematics, Sun Yat-Sen University, Guangzhou, China

    Jiashu Huang, Jun Tan & Ning Bi

Authors
  1. Jiashu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiashu Huang .

Editor information

Editors and Affiliations

  1. East China Normal University, Shanghai, China

    Yue Lu

  2. Paris Descartes University, Paris, France

    Nicole Vincent

  3. Hong Kong Baptist University, Kowloon, Hong Kong

    Pong Chi Yuen

  4. Sun Yat-sen University, Guangzhou, China

    Wei-Shi Zheng

  5. Polytechnique Montréal, Montreal, QC, Canada

    Farida Cheriet

  6. Concordia University, Montreal, QC, Canada

    Ching Y. Suen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, J., Tan, J., Bi, N. (2020). Overview of Mathematical Expression Recognition. In: Lu, Y., Vincent, N., Yuen, P.C., Zheng, WS., Cheriet, F., Suen, C.Y. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2020. Lecture Notes in Computer Science(), vol 12068. Springer, Cham. https://doi.org/10.1007/978-3-030-59830-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59830-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59829-7

  • Online ISBN: 978-3-030-59830-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics