Skip to main content
SpringerLink
Log in

All-content text recognition method for financial ticket images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the development of the economy, the number of financial tickets is increasing. The traditional invoice reimbursement and entry work bring more and more burden to financial accountants. However, standard OCR technology weakly supports financial tickets with various layouts and mixed Chinese and English characters. In view of this problem, this paper designs a method of financial ticket all-content text information detection and recognition based on deep learning. This method can effectively suppress the common noise of ticket image and extract financial information from ticket image in batch. At the same time, aiming at the problem of multi-character mixed character recognition, we propose a financial ticket character recognition framework (FTCRF), which can improve the accuracy of multi-character mixed character recognition and make the detection and recognition of financial ticket surface information more efficient. The experimental results show that the average recognition accuracy of the character sequence is 91.75%. The average recognition accuracy of the whole ticket is 87%, which significantly improves the efficiency of the financial accounting system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Awad AI, Hassaballah M (2016) Image feature detectors and descriptors. Studies in Computational Intelligence Springer International Publishing, Cham

    Book  Google Scholar 

  2. Charniak E (2019) Introduction to deep learning. The MIT Press

  3. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:14123555

  4. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893

  5. Deng D, Liu H, Li X, Cai D (2018) Pixellink: Detecting scene text via instance segmentation. arXiv preprint arXiv:180101315

  6. Feng X, Jiang Y, Yang X, Du M, Li X (2019) Computer vision algorithms and hardware implementations: A survey. Integration 69:309–320

    Article  Google Scholar 

  7. Ha HT (2017) Recognition of invoices from scanned documents. In: RASLAN, pp 71–78

  8. Hassaballah M, Awad AI (2016) Detection and description of image features: An introduction. In: Image feature detectors and descriptors, Springer, pp 1–8

  9. Hassaballah M, Awad AI (2020) Deep Learning in Computer Vision: Principles and Applications. Deep Learning in Computer Vision: Principles and Applications

  10. Hassaballah M, Hosny K M (2019) Recent advances in computer vision. Studies in Computational Intelligence 804

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  12. He K, Gkioxari G, Dollár P, Girshick R (2017a) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  13. He P, Huang W, He T, Zhu Q, Qiao Y, Li X (2017b) Single shot text detector with regional attention. In: Proceedings of the IEEE international conference on computer vision, pp 3047–3055

  14. He W, Zhang XY, Yin F, Liu CL (2017c) Deep direct regression for multi-oriented scene text detection. In: Proceedings of the IEEE international conference on computer vision, pp 745–753

  15. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  16. Jaderberg M, Simonyan K, Vedaldi A, Zisserman A (2016) Reading text in the wild with convolutional neural networks. Int J Comput Vis 116 (1):1–20

    Article  MathSciNet  Google Scholar 

  17. Jha M, Kabra M, Jobanputra S, Sawant R (2019) Automation of cheque transaction using deep learning and optical character recognition, IEEE

  18. Liao M, Shi B, Bai X, Wang X, Liu W (2016) Textboxes: A fast text detector with a single deep neural network. arXiv preprint arXiv:161106779

  19. Liao M, Shi B, Bai X (2018) Textboxes++: A single-shot oriented scene text detector. IEEE Trans Image Process 27(8):3676–3690

    Article  MathSciNet  Google Scholar 

  20. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37

  21. Lowe D G (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  22. Miikkulainen R, Liang J, Meyerson E, Rawal A, Fink D, Francon O, Raju B, Shahrzad H, Navruzyan A, Duffy N et al (2019) Artificial intelligence in the age of neural networks and brain computing

  23. Palm R B, Winther O, Laws F (2017) Cloudscan-a configuration-free invoice analysis system using recurrent neural networks. In: 2017 14Th IAPR international conference on document analysis and recognition (ICDAR), vol 1. IEEE, pp 406–413

  24. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  25. Shi B, Bai X, Yao C (2016) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2298–2304

    Article  Google Scholar 

  26. Shi C, Wang C, Xiao B, Zhang Y, Gao S, Zhang Z (2013) Scene text recognition using part-based tree-structured character detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2961–2968

  27. Shreya S, Upadhyay Y, Manchanda M, Vohra R, Singh GD (2019) Optical character recognition using convolutional neural network. In: 2019 6Th international conference on computing for sustainable global development (INDIACom), IEEE, pp 55–59

  28. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556

  29. Solis AI, Nava P (2019) Domain specific architectures, hardware acceleration for machine/deep learning. In: Disruptive technologies in information sciences II, International Society for Optics and Photonics, vol 11013, pp 1101307

  30. Srivastava S, Priyadarshini J, Gopal S, Gupta S, Dayal HS (2019) Optical character recognition on bank cheques using 2d convolution neural network. In: Applications of Artificial Intelligence Techniques in Engineering, Springer, pp 589–596

  31. Sun Y, Mao X, Hong S, Xu W, Gui G (2019) Template matching-based method for intelligent invoice information identification. IEEE Access 7:28392–28401

    Article  Google Scholar 

  32. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  33. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  34. Tan M, Le QV (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:190511946

  35. Wang W, Xie E, Li X, Hou W, Lu T, Yu G, Shao S (2019) Shape robust text detection with progressive scale expansion network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9336–9345

  36. Wang Y, Morariu VI, Davis LS (2018) Learning a discriminative filter bank within a cnn for fine-grained recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4148–4157

  37. Xie E, Zang Y, Shao S, Yu G, Yao C, Li G (2019) Scene text detection with supervised pyramid context network. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 9038–9045

  38. Yi F, Zhao Y F, Sheng G Q, Xie K, Wen C, Tang X G, Qi X (2019) Dual model medical invoices recognition. Sensors 19(20):4370

    Article  Google Scholar 

  39. Zhang Z, Zhang C, Shen W, Yao C, Liu W, Bai X (2016) Multi-oriented text detection with fully convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4159–4167

  40. Zhou X, Yao C, Wen H, Wang Y, Zhou S, He W, Liang J (2017) East: An efficient and accurate scene text detector. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5551–5560

Download references

Acknowledgements

This research was partially supported by the National Science Foundation of China under Grant Nos. 62050194, 62037001, 61721002, and 62002282, the MOE Innovation Research Team No. IRT_17R86, and Project of XJTU-SERVYOU Joint Tax-AI Lab.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, China

    Hanning Zhang, Qinghua Zheng & Boqin Feng

  2. Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Technology Research and Development, Xi’an Jiaotong University, Xi’an, 710049, China

    Hanning Zhang

  3. School of Continuing Education, Xi’an Jiaotong University, Xi’an, 710049, China

    Bo Dong

  4. National Engineering Lab for Big Data Analytics, Xi’an Jiaotong University, Xi’an, 710049, China

    Bo Dong

  5. Xi’an Network Computing Data Technology Co., Ltd., Xi’an, 710049, China

    Bo Xu & Haiyu Wu

Authors
  1. Hanning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinghua Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Boqin Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haiyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Dong.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Dong, B., Zheng, Q. et al. All-content text recognition method for financial ticket images. Multimed Tools Appl 81, 28327–28346 (2022). https://doi.org/10.1007/s11042-022-12741-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12741-2

Keywords

Search

Navigation