Skip to main content
SpringerLink
Log in

Intelligent Character Recognition of Handwritten Forms with Deep Neural Networks

  • Conference paper
  • First Online:
Towards AI-Aided Invention and Innovation (TFC 2023)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 682))

Included in the following conference series:

  • 452 Accesses

Abstract

The automatic processing of handwritten forms remains a challenging task, wherein detection and subsequent classification of handwritten characters are essential steps. We describe a novel approach, in which both steps - detection and classification - are executed in one task through a deep neural network. Therefore, training data is not annotated by hand, but manufactured artificially from the underlying forms and yet existing datasets. It can be demonstrated that this single-task approach is superior in comparison to the state-of-the-art two task approach. The current study focuses on hand-written Latin letters and employs the EMNIST data set. However, limitations were identified with this data set, necessitating further customization. Finally, an overall recognition rate of 88.28% was attained on real data obtained from a written exam.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Barchard, K.A., Pace, L.A.: Preventing human error: The impact of data entry methods on data accuracy and statistical results. Comput. Hum. Behav. 27, 1834–1839 (2011). https://doi.org/10.1016/j.chb.2011.04.004

    Article  Google Scholar 

  2. Adriano, J.E.M., Calma, K.A.S., Lopez, N.T., Parado, J.A., Rabago, L.W., Cabardo, J.M.: Digital conversion model for hand-filled forms using optical character recognition (OCR). IOP Conf. Ser. Mater. Sci. Eng. 482, 012049 (2019). https://doi.org/10.1088/1757-899X/482/1/012049

    Article  Google Scholar 

  3. Rasmussen, L.V., Peissig, P.L., McCarty, C.A., Starren, J.: Development of an optical character recognition pipeline for handwritten form fields from an electronic health record. J. Am. Med. Inform. Assoc. 19, e90–e95 (2012). https://doi.org/10.1136/amiajnl-2011-000182

    Article  Google Scholar 

  4. Jha, M., Kabra, M., Jobanputra, S., Sawant, R.: Automation of cheque transaction using deep learning and optical character recognition. In: 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, pp. 309–312. IEEE (2019). https://doi.org/10.1109/ICSSIT46314.2019.8987925

  5. Deodhare, D., Suri, N.R., Amit, R.: Preprocessing and image enhancement algorithms for a form-based intelligent character recognition system. Comput. Sci. 2, 131–144 (2005)

    Google Scholar 

  6. Islam, N., Islam, Z., Noor, N.: A survey on optical character recognition system. J. Inf. 10 (2016)

    Google Scholar 

  7. Somashekar, T.: A Survey on handwritten character recognition using machine learning technique. J. Univ. Shanghai Sci. Technol. 23, 1019–1024 (2021). https://doi.org/10.51201/JUSST/21/05304. Research Scholar, Department of Computer Science & Engineering, University College of Engineering(A), Osmania, University, Hyderabad, Telangana, India

    Article  Google Scholar 

  8. Khobragade, R.N., Koli, N.A., Lanjewar, V.T.: Challenges in recognition of online and off-line compound handwritten characters: a review. In: Zhang, Y.-D., Mandal, J.K., So-In, C., Thakur, N.V. (eds.) Smart Trends in Computing and Communications, vol. 165, pp. 375–383. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-0077-0_38

    Chapter  Google Scholar 

  9. Priya, A., Mishra, S., Raj, S., Mandal, S., Datta, S.: Online and offline character recognition: a survey. In: 2016 International Conference on Communication and Signal Processing (ICCSP), Melmaruvathur, Tamilnadu, India, pp. 0967–0970. IEEE (2016). https://doi.org/10.1109/ICCSP.2016.7754291

  10. Suriya, S., Dhivya, S., Balaji, M.: Intelligent character recognition system using convolutional neural network. EAI Endorsed Trans. Cloud Syst. 6, 166659 (2020). https://doi.org/10.4108/eai.16-10-2020.166659

    Article  Google Scholar 

  11. Memon, J., Sami, M., Khan, R.A., Uddin, M.: Handwritten optical character recognition (OCR): a comprehensive systematic literature review (SLR). IEEE Access 8, 142642–142668 (2020). https://doi.org/10.1109/ACCESS.2020.3012542

    Article  Google Scholar 

  12. Nath, G.: Isolated OCR for handwritten forms: an application in the education domain. In: 8th International Conference of Business Analytics and (2022)

    Google Scholar 

  13. Pal, A., Singh, D.: Handwritten English character recognition using neural network (2010)

    Google Scholar 

  14. Khobragade, R.N., Koli, D.N.A., Makesar, M.S.: A survey on recognition of Devnagari script. Int. J. Comput. Appl. (2013)

    Google Scholar 

  15. Singh, S., Tiwari, S.: Application of image processing and convolution networks in intelligent character recognition for digitized forms processing. Int. J. Comput. Appl. 179, 7–13 (2018). https://doi.org/10.5120/ijca2018915460

    Article  Google Scholar 

  16. Rao, N.V., Sastry, A.S.C.S., Chakravarthy, A.S.N., Kalyanchakravarthi, P.: Optical character recognition technique algorithms. J. Theor. Appl. Inf. Technol. 83 (2016)

    Google Scholar 

  17. Khorsheed, M.S.: Off-line arabic character recognition – a review. Pattern Anal. Appl. 5(1), 31–45 (2002). https://doi.org/10.1007/s100440200004

    Article  MathSciNet  Google Scholar 

  18. Patil, P.M., Sontakke, T.R.: Rotation, scale and translation invariant handwritten Devanagari numeral character recognition using general fuzzy neural network. Pattern Recognit. 40, 2110–2117 (2007). https://doi.org/10.1016/j.patcog.2006.12.018

    Article  MATH  Google Scholar 

  19. KumarShrivastava, S., Chaurasia, P.: Handwritten Devanagari lipi using support vector machine. Int. J. Comput. Appl. 43, 20–25 (2012). https://doi.org/10.5120/6220-8785

    Article  Google Scholar 

  20. Ngo, P.: Digital line segment detection for table reconstruction in document images. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds.) ICIAP 2022. LNCS, vol. 13232, pp. 211–224. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06430-2_18

    Chapter  Google Scholar 

  21. Gatos, B., Danatsas, D., Pratikakis, I., Perantonis, S.J.: Automatic table detection in document images. In: Singh, S., Singh, M., Apte, C., Perner, P. (eds.) ICAPR 2005. LNCS, vol. 3686, pp. 609–618. Springer, Heidelberg (2005). https://doi.org/10.1007/11551188_67

    Chapter  Google Scholar 

  22. Green, E., Krishnamoorthy, M.: Model-based analysis of printed tables. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, Montreal, Que., Canada, pp. 214–217. IEEE Computer Society Press (1995). https://doi.org/10.1109/ICDAR.1995.598979

  23. Alhéritière, H., Amaïeur, W., Cloppet, F., Kurtz, C., Ogier, J.-M., Vincent, N.: Straight line reconstruction for fully materialized table extraction in degraded document images. In: Couprie, M., Cousty, J., Kenmochi, Y., Mustafa, N. (eds.) DGCI 2019. LNCS, vol. 11414, pp. 317–329. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14085-4_25

    Chapter  MATH  Google Scholar 

  24. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 580–587. IEEE (2014). https://doi.org/10.1109/CVPR.2014.81

  25. Girshick, R.: Fast R-CNN (2015). http://arxiv.org/abs/1504.08083

  26. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  27. Li, W., Feng, X.S., Zha, K., Li, S., Zhu, H.S.: Summary of target detection algorithms. J. Phys. Conf. Ser. 1757, 012003 (2021). https://doi.org/10.1088/1742-6596/1757/1/012003

    Article  Google Scholar 

  28. Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: TableBank: a benchmark dataset for table detection and recognition (2019). https://doi.org/10.48550/ARXIV.1903.01949

  29. Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: CascadeTabNet: an approach for end to end table detection and structure recognition from image-based documents (2020). https://doi.org/10.48550/ARXIV.2004.12629

  30. Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022). http://arxiv.org/abs/2207.02696

  31. Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2. https://github.com/facebookresearch/detectron2. Accessed 05 Apr 2023

  32. Raj, S., Gupta, Y., Malhotra, R.: license plate recognition system using YOLOv5 and CNN. In: 2022 8th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, pp. 372–377. IEEE (2022). https://doi.org/10.1109/ICACCS54159.2022.9784966

  33. Shi, H., Zhao, D.: License plate recognition system based on improved YOLOv5 and GRU. IEEE Access. 11, 10429–10439 (2023). https://doi.org/10.1109/ACCESS.2023.3240439

    Article  Google Scholar 

  34. Tang, M., Xie, S., He, M., Liu, X.: Character recognition in endangered archives: Shui manuscripts dataset, detection and application realization. Appl. Sci. 12, 5361 (2022). https://doi.org/10.3390/app12115361

    Article  Google Scholar 

  35. Goswami, R., Sharma, O.P.: A review on character recognition techniques. Int. J. Comput. Appl. 83, 18–23 (2013). https://doi.org/10.5120/14460-2737

    Article  Google Scholar 

  36. Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: EMNIST: extending MNIST to handwritten letters. In: 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, pp. 2921–2926. IEEE (2017). https://doi.org/10.1109/IJCNN.2017.7966217

  37. Image Classification on EMNIST-Letters. https://paperswithcode.com/sota/image-classification-on-emnist-letters. Accessed 05 Apr 2023

  38. Jeevan, P., Viswanathan, K., Sethi, A.: WaveMix: a resource-efficient neural network for image analysis (2023). http://arxiv.org/abs/2205.14375

  39. Kabir, H.M.D., et al.: SpinalNet: deep neural network with gradual input (2022). http://arxiv.org/abs/2007.03347

  40. Jayasundara, V., Jayasekara, S., Jayasekara, H., Rajasegaran, J., Seneviratne, S., Rodrigo, R.: TextCaps: handwritten character recognition with very small datasets. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 254–262 (2019). https://doi.org/10.1109/WACV.2019.00033

  41. Gesmundo, A.: A continual development methodology for large-scale multitask dynamic ML systems (2022). http://arxiv.org/abs/2209.07326

  42. Pad, P., Narduzzi, S., Kundig, C., Turetken, E., Bigdeli, S.A., Dunbar, L.A.: Efficient neural vision systems based on convolutional image acquisition. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 12282–12291. IEEE (2020). https://doi.org/10.1109/CVPR42600.2020.01230

  43. Shawon, A., Jamil-Ur Rahman, Md., Mahmud, F., Arefin Zaman, M.M.: Bangla handwritten digit recognition using deep CNN for large and unbiased dataset. In: 2018 International Conference on Bangla Speech and Language Processing (ICBSLP), Sylhet, pp. 1–6. IEEE (2018). https://doi.org/10.1109/ICBSLP.2018.8554900

  44. Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: 2011 International Conference on Document Analysis and Recognition, Beijing, China, pp. 1135–1139. IEEE (2011). https://doi.org/10.1109/ICDAR.2011.229

  45. Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: EMNIST: an extension of MNIST to handwritten letters (2017). http://arxiv.org/abs/1702.05373

  46. Smith, R.: An overview of the tesseract OCR engine. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), Curitiba, Parana, Brazil, vol 2. pp. 629–633. IEEE (2007). https://doi.org/10.1109/ICDAR.2007.4376991

Download references

Author information

Authors and Affiliations

  1. Offenburg University of Applied Sciences, Badstraße 24, 77652, Offenburg, Germany

    Hartwig Grabowski

Authors
  1. Hartwig Grabowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hartwig Grabowski .

Editor information

Editors and Affiliations

  1. ICube-CSIP/INSA Strasbourg, Strasbourg, France

    Denis Cavallucci

  2. Offenburg University of Applied Sciences, Offenburg, Germany

    Pavel Livotov

  3. Technical University of Cluj-Napoca, Cluj-Napoca, Romania

    Stelian Brad

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grabowski, H. (2023). Intelligent Character Recognition of Handwritten Forms with Deep Neural Networks. In: Cavallucci, D., Livotov, P., Brad, S. (eds) Towards AI-Aided Invention and Innovation. TFC 2023. IFIP Advances in Information and Communication Technology, vol 682. Springer, Cham. https://doi.org/10.1007/978-3-031-42532-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42532-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42531-8

  • Online ISBN: 978-3-031-42532-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation