Abstract
Receipts are crucial for many businesses’ operation, where expenses are tracked meticulously. Receipt documents are often scanned into images, digitized and analyzed before the information is streamed into institutional financial applications. The precise extraction of expense data from receipt images is a difficult task owed to the high variance in fonts and layouts, the frailty of the print paper, unstructured scanning environments and an immeasurable amount of domains. We propose a method that combines visual and linguistic features for automatic information retrieval from receipt images using deep network architectures, which outperforms existing approaches. Our Skip-Rect Embedding (SRE) descriptor is demonstrated in two canonical applications for receipt information retrieval: field extraction and Optical Character Recognition (OCR) error enhancement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Based on experiments we performed using the prolific Tesseract OCR engine on preprocessed, phone-captured images [19].
References
Aslan, E., Karakaya, T., Unver, E., Akgül, Y.S.: A part based modeling approach for invoice parsing. In: VISIGRAPP (3: VISAPP), pp. 392–399 (2016)
Cao, K., Rei, M.: A joint model for word embedding and word morphology. arXiv preprint arXiv:1606.02601 (2016)
Cesarini, F., Francesconi, E., Gori, M., Soda, G.: Analysis and understanding of multi-class invoices. Doc. Anal. Recogn. 6(2), 102–114 (2003)
Chen, K., Seuret, M., Liwicki, M., Hennebert, J., Ingold, R.: Page segmentation of historical document images with convolutional autoencoders. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1011–1015. IEEE (2015)
Hamza, H., Belaïd, Y., Belaïd, A.: Case-based reasoning for invoice analysis and recognition. In: Weber, R.O., Richter, M.M. (eds.) ICCBR 2007. LNCS (LNAI), vol. 4626, pp. 404–418. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74141-1_28
Jaech, A., Mulcaire, G., Ostendorf, M., Smith, N.A.: A neural model for language identification in code-switched tweets. In: Proceedings of the Second Workshop on Computational Approaches to Code Switching, pp. 60–64 (2016)
Janssen, B., Saund, E., Bier, E., Wall, P., Sprague, M.A.: Receipts2Go: the big world of small documents. In: Proceedings of the 2012 ACM symposium on Document Engineering, pp. 121–124. ACM (2012)
Johnson, J., Karpathy, A., Fei-Fei, L.: DenseCap: fully convolutional localization networks for dense captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4565–4574 (2016)
Kang, L., Kumar, J., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for document image classification. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 3168–3172. IEEE (2014)
Klein, B., Agne, S., Dengel, A.: Results of a study on invoice-reading systems in Germany. In: Marinai, S., Dengel, A.R. (eds.) DAS 2004. LNCS, vol. 3163, pp. 451–462. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28640-0_43
Lu, C., Krishna, R., Bernstein, M., Fei-Fei, L.: Visual relationship detection with language priors. arXiv:1608.00187 [cs], July 2016
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv:1301.3781 [cs, stat], January 2013
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. arXiv:1310.4546 [cs, stat], October 2013
Palm, R.B., Winther, O., Laws, F.: CloudScan-a configuration-free invoice analysis system using recurrent neural networks. arXiv preprint arXiv:1708.07403 (2017)
Pennington, J., Socher, R., D. Manning, C.: GloVe: global vectors for word representation. (2014). https://nlp.stanford.edu/pubs/glove.pdf
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Santos, C.D., Zadrozny, B.: Learning character-level representations for part-of-speech tagging. In: Proceedings of the 31st International Conference on Machine Learning (ICML 2014), pp. 1818–1826 (2014)
Schulz, F., Ebbecke, M., Gillmann, M., Adrian, B., Agne, S., Dengel, A.: Seizing the treasure: transferring layout knowledge in invoice analysis. In: 10 International Conference on Document Analysis and Recognition (ICDAR), Barcelona, Spain (2009)
Smith, R.: An overview of the Tesseract OCR engine. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition, ICDAR 2007, vol. 02, pp. 629–633. IEEE Computer Society, Washington, DC, USA (2007). http://dl.acm.org/citation.cfm?id=1304596.1304846
Sorio, E., Bartoli, A., Davanzo, G., Medvet, E.: Open world classification of printed invoices. In: Proceedings of the 10th ACM symposium on Document engineering, pp. 187–190. ACM (2010)
Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway Networks. arXiv:1505.00387, May 2015
Vosoughi, S., Vijayaraghavan, P., Roy, D.: Tweet2vec: learning Tweet embeddings using character-level CNN-LSTM encoder-decoder. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 1041–1044. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2911451.2914762, http://doi.acm.org/10.1145/2911451.2914762
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gal, R., Morag, N., Shilkrot, R. (2019). Visual-Linguistic Methods for Receipt Field Recognition. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-20890-5_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20889-9
Online ISBN: 978-3-030-20890-5
eBook Packages: Computer ScienceComputer Science (R0)